Bootstrapping Knowledgeabout Social Phenomenausing Simulation Models

Bruce Edmondshttp://bruce.edmonds.name

Centre for Policy Modelling

http://cfpm.org

The purposes of this talk are…

• To examine the role of simulation models in the understanding of social phenomena– (i.e. not entertainment, illustration, art etc.)

• Remind people how difficult the task is!• To criticise both naïve positivist and relativistic

post-modernist positions• To argue against models that fail to meet any

set of reasonable criteria – ‘floating models’

• To sketch a way forward for social simulation as a messy but evolving process

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 2

slide-2

Difficulties of social simulation

• The complication and complexity of social phenomena (more than biology)

• The difficulty of building, checking and understanding simulations

• The sheer inadequacy of formal models for representing rich phenomena

• The lack of rich and multifaceted data about social phenomena

• The many assumptions not supported by evidence in any social simulation......e.g. how the parts that are being represented

behave (compared to, say, chemistry)!

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 3

slide-3

The Astounding Assumption

That we can make useful computational models of social

phenomena...

...even though we know that many of the details in our models are

wrong!

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 4

slide-4

Despite these difficulties...

• There are those that hope to find models that are both simple and useful

(the optimistic, Nobel-prize-winning scenario)

• By:– Looking for a very clever ‘fundamental’ behind

broad classes of social phenomena– Simply hoping that the details will not happen to

matter:• On average when many agents are interacting• Or over the long-run

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 5

slide-5

Justifications for such optimism

Such hopes are (at times) justified by appeals to:• Simplicity – that simpler models are more

likely to be truth (or truth-like etc.)• The Law of Large Numbers – that the ‘noise’

will cancel out en masse (i.e. is random)• Abstraction – that abstracting from detail will

result in greater generality• Plausibility – that an academic’s intuitions are

sufficient to ensure relevance• Data Fitting – that the model outcomes

vaguely match that of some data

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 6

slide-6

Rather it looks like that...

• There will be no ‘short-cut’ to useful models of social ‘systems’ that reliably refer to observations of these systems

• Social phenomena is at least as complex as biological phenomena but with not evidence that there is a single structuring process (like that of the Darwinian-synthesis)

• What appears to be promising simple models will turn out to be nothing more than computational analogies (at least until a lot more empirical work and modelling is done)

• When it does appear, theory will be mundane and specific and not a ‘grand theory’

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 7

slide-7

A Picture of a Modelling Relation

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 8

slide-8

Simulation run

enco

ding

or

mea

sure

men

t decoding or prediction

Target Process

A Picture of an Attenuated Modelling Relation

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 9

slide-9

Simulation runen

codi

ng o

r m

easu

rem

ent

decoding or prediction

Target Process

A Messy ‘Evolution’ of knowledge

• Useful knowledge might be ‘evolved’ • But it will be a ‘mess’ of bits• Each useful in specific ‘niches’• Not determined by the evidence of the

niches but sufficient and competitive there• Gradually increasing the ‘fit’ to the ‘niches’

based on previous knowledge• Compatible with a rich but context-specific

range of evidence (including personal)• (Co-evolving with new data collection)

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 10

slide-10

But this depends on...

• There being a formal replicator or ‘back-bone’ for the evolutionary process to act on (i.e. the formal models)

• A ‘Cambrian Explosion’ of specific representations are allowed to adapt in complex and specific ways (different purposes, groups, etc.) (simulation)

• There is sufficient ‘selective pressure’ from rich evidence to drive the process so that ‘toy models’ are eliminated/refined

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 11

slide-11

Can this process be progressive (in any sense) or only relativistic?

• Neither completely!• Social processes, institutions and purposes do

change fairly rapidly so this will continually require new kinds of knowledge and models

• But if models can be compared as to which possibilities are more likely (or which are ruled out) to some extent then localised progress is possible

• If some restriction or focus of the suggested possibilities can be established even if ‘only’ about specific context, purpose, assumptions

• Allowing a bootstrapping process of increasing reliability, starting from very fallible knowledge

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 12

slide-12

Analogy with the development of measurement• From crude, intuitive measures (length of

King’s foot) to sophisticated measuring technologies (SI in advanced countries)

• More reliability when accompanying theory is developed but considerable progress with no or only ‘mundane’ theory

• Initial uncertainty slowly used to produce more accurate measurement methods

• Measurement accuracy is boot-strapped over time based on previous measurement

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 13

slide-13

Kinds of modelling that are conducive to such a process

1. Probing: Modelling that tests, reveals and documents model assumptions and weaknesses

2. Evidential: Modelling that finds and strengthens relationships with evidence (including stakeholder opinion) – e.g. Explanatory, predictive, instrumental

3. Pseudo-maths: revealing such general and important model patterns that this will be systematically applicable to other models

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 14

slide-14

Evidence-Relevant Modelling

• Includes models with an explanatory purpose or a predictive purpose

• But not those that fail to be any!– Not explanatory because they use overly strong

assumptions (encoding is weak)– Not predictive because they are not tested on

unknown data (decoding is weak)– Not shown to be effective because they are not

used by stakeholders

• Includes evidence in the form of stakeholder’s opinions and narratives (as in ComMod approach)

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 15

slide-15

The problem is that there are models that do none of these

• A plethora of ‘floating’ models that are:– Motivated according to the intuitions of

the modeller but use strong assumptions

– Fitted known evidence of outcomes

– Not general enough to be pseudo-maths

• Fail against any of the mentioned sets of criteria

• Are more like computational analogies

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 16

slide-16

A Picture of a Computational Analogy

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 17

slide-17

Simulation runen

codi

ng o

r m

easu

rem

ent decoding or

prediction

A way of thinking about the Target Phenomena

Personal vs. Scientific Knowledge

• Playing with a model can usefully inform the intuitions of the modeller and users

• Including playing with floating models• But what is learnt is personal knowledge• Which is not necessarily something

transferable to others in another context• (The model can be transferred but there is

almost no selection against the evidence)• (The ideas can be useful in helping to

understand phenomena but requires embedding in rich expert knowledge base)

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 18

slide-18

Sufficient and appropriate selective pressure on models is very important

• Otherwise there is no adaptive evolution (we already have lots of model variety)

• To be maximally relevant to observed systems selective pressure has to be from these niches(rather than, say, fun to play with)

• Analogies and ideas are important but they are not the evolutionary back-bone

• Surviving models will probably not be simple or have wide scope

• But will be those that say something about the evidence (however specific and conditional)

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 19

slide-19

Some Possible Directions for Social Simulation

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 20

slide-20

Computational analogies that come and go

A retreat into purely formal explorations

An evolving mess of specific and

complex models that relate to observations

Simple floating models that fail to predict

or explain

Conclusion

• The bootstrapping of useful and reliable knowledge of social phenomena is feasible

• Using the evolution of computational simulations across a body of researchers

• But only if these simulations are adapted to social phenomena via rich evidence

• Most importantly that selective pressure (i.e. criticism) is applied to ‘floating models’

• But it will be a ‘mess’ of quite specific and context-dependent knowledge

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 21

slide-21

Postcript – after the philosophy

• If we think that there is a role for simulation in the understanding of social processes...

• more than can be obtained through natural language discourse and analogies alone...

• and we think that the epistemological problems of simulation are intertwined with the social processes of science

• Then surely we have to simulate the social processes of science and knowledge development and not only discuss it

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 22

slide-22

Gratuitous Advert – ESSA 2009

Special stream for papers about “Simulating the Social Processes of Science”

Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 23

slide-23