andy stirling - nexus methods
TRANSCRIPT
‘Nexus Methods’
opening up democracy as rigour
Andy StirlingSPRU & STEPS Centre
University of Sussex
presentation to conference session on Nexus Methods ESRC Methods Festival
University of Bath5th July 2016
www.steps-centre.org/www.sussex.ac.uk/spru/www.multicriteriamapping.com www.sussex.ac.uk/spru/people/peoplelists/person/7513
How Shall We Play This?
1000
1130
1100
1130
1200
1230
STEPSMethods talk then discussion Andy Stirling
Open Spacevolunteersconvene groups
Ses
sion
1br
eak
Ses
sion
2
Nexus Methodology talk then discussionAndy Stirling
Open Spacevolunteersconvene groups
STEPSMethods Andy Stirling
OPTION 3OPTION 1 OPTION 2
What is ‘the Nexus’?
food
water
energy
Substantive imperatives for ‘joined up policy…?
… or instrumental pressure for policy justification?
What is ‘the Nexus’?climate
developmentfood
environment
water
security
globalisationenergy
population
migrationurbanisation
Phenonema Under Scrutiny
• social and material world
•“systems” and “contexts”
• “scales” and “levels”
• “actors” and “networks”
• “values” and “interests”
• “frames” and “narratives”
• “causes” and “effects”
• processes and relations
• “knowledges”,“incertitudes”
• “positives” and “negatives”
• “structures” and “agents”
• “actions” and “reactions”
• “imaginations” and “visions”
• “metrics” and ”indices”
Complexity is not made by pathways methods…
…but addressed by them – brings a need for “reflexivity”
cost-benefit analysisrisk assessmentdecision analysismultiattribute utility theory technology assessmentlife cycle analysisoptimisation modellingBayesian networksextended accountingdata miningdelphi methods
Quantitative Culture Qualitative Culture
A Juicy Opportunity for Appraisal Disciplines!
No shortage of candidate ‘Nexus methods’
scenario workshopsfocus groups
participatory appraisalstakeholder deliberation
ethnomethodologycollaborative design
capabilities assessment strategic appraisal
action researchcooperative research
study groups
Energy regulation: most mature, sophisticated comparative analysis…
A Key Common Problem: Concealed Ambiguity
0.001 0.1 10 1000externality’: cUS/kWh (after Sundqvist et al, 2005)low RISK high
coal
oil
gas
nuclear
hydro
wind
solar
biomass
Energy regulation: most mature, sophisticated comparative analysis…
A Key Common Problem: Concealed Ambiguity
0.001 0.1 10 1000
coal
oil
gas
nuclear
hydro21
wind
solar
biomass
n =
‘externality’: cUS/kWh (after Sundqvist et al, 2005)
minimum maximum25% 75%
low RISK high
Energy regulation: most mature, sophisticated comparative analysis…
A Key Common Problem: Concealed Ambiguity
coal
oil
gas
nuclear
hydro
36
20
wind 18
solar 11
biomass 22
31
21
16
n =
…‘sound science’, ‘evidence based’ Nexus analyses justify many policies
Energy regulation: most mature, sophisticated comparative analysis…
A Key Common Problem: Concealed Ambiguity
All Quantification is Qualitatively FramedEqually true of qualitative research, but this is better appreciated
‘scope’
‘system’‘focus’
‘cause’
‘effect’
under-determined realities picture of problem/solution
Framing ‘the Nexus’
environment‘scope’
‘system’‘focus’
subjective framing
‘effect’
under-determined realities
‘cause’view 1
picture of problem/solution
Framing ‘the Nexus’
environment
‘system’
under-determined realities diverse picturesplural frames
‘effect’
‘cause’
‘scope’
‘focus’
view 2
view 1
conte
nding
know
ledge
s
and v
alues
Framing ‘the Nexus’
plural frames
‘system’
‘cause’
‘effect’
under-determined realities diverse pictures
‘scope’
‘focus’
view 3
view 1
view 2
conte
nding
know
ledge
s
and v
alues
Framing ‘the Nexus’
view 4
plural frames
‘system’
‘cause’
‘effect’
under-determined realities diverse pictures
‘scope’
‘focus’
conte
nding
know
ledge
s
and v
alues
view 1
view 2
view 3
Framing ‘the Nexus’
incumben
ts
power in knowledge
‘system’
‘cause’
‘effect’
under-determined realities
academic
government
local people
diverse pictures
‘scope’
‘focus’
conte
nding
know
ledge
s
and v
alues
Framing ‘the Nexus’
“THE NEXUS”
fundamental nature
biophysical environments
societies & economies
cultures &
institutions
Science (of all kinds!) is a key means by
which to remind “the real world”
of politics about the
“real real world” of everything else
The Conditionality of Assessment
The Conditionality of Assessment
knowledge practices
fundamental nature
biophysical environments
societies & economies
cultures & instituions
But knowledge is produced by people,
with cultures, pursuing practices
in institutions
So it is jointly shaped to reflect
both social and natural orders
KNOWLEDGE PRACTICES
knowledge practices
fundamental nature
biophysical environments
societies & economies
INSTITUTIONS
The Conditionality of Assessment
This leads to a paradox
What is inside…… is also outside
knowledge practices
biophysical environments
societies & economies
INSTITUTIONS
The Knowing-Known Dualism in Assessment
This leads to a paradox
What is inside…… is also outside
… as in a Klein Bottle, knowledge is on the
inside and the outside of human action
KNOWLEDGE PRACTICES
reflexivity over subjectivity, contingency, contextuality
refle
ctio
n ov
er ro
bust
ness
, qua
lity,
err
or‘Reflexivity’ is not about ‘Anything Goes’
normatively and/or epistemically wrong
normatively and/or epistemically right
CONCEPTUAL SPACE OF ALL CANDIDATE POSSIBILITIES
FOR WHAT IS NORMATIVELY OR EPISTEMICALLY ‘RIGHT’
KEY
normatively and/or epistemically wrong
normatively and/or epistemically right
naïve realism
“one thing is precisely right”
mainstream fallibilism
“one thing is approximately right”
caricature relativism
“all things are equally right”
conditional plurality
“contrasting things are equally right… xxxx…but much else is just plain wrong!”
reflexivity over subjectivity, contingency, contextuality
refle
ctio
n ov
er ro
bust
ness
, qua
lity,
err
or‘Reflexivity’ is not about ‘Anything Goes’
reflexivity: ‘right’ answers & solutions vary by framings of questions & problems
molecular biologists
ecologists economists
sociologists
Review of >100 different kinds of ‘Nexus-related method’
Background Approach
- epistemic cultures: quantitative / hybrid / qualitative
- styles of reasoning: deductive / inductive / abductive
- relations & practices: analytic / interactive
- procedural functions: frameworks / techniques / tools
RESEARCH METHODSprimarily qualitative mostly quantitative
EPISTEMIC CULTUREexplicitly mixed
MO
DE
OF
ENQ
UIR
Yla
rgel
y de
duct
ive
chie
fly a
bduc
tive
mai
nly
indu
ctiv
eRESEARCH METHODS
QUALITATIVE APPREHENSION HYBRID APPRAISAL EMPIRICAL QUANTIFICATION
GROUNDED METHODS
INTERPRETIVE ANALYSIS QUANTITATIVE ASSESSMENT
APPRECIATIVE APPROACHES
MIXED THEORY
EVALUATIVE TECHNIQUES
integrated assessment
largely qualitative mostly quantitative EPISTEMIC CULTURE
explicitly mixed
STYL
E O
F EN
QU
IRY
dedu
ctiv
e st
yle
abdu
ctiv
e st
yle
indu
ctiv
e st
yle QUALITATIVE APPREHENSION HYBRID APPRAISAL EMPIRICAL QUANTIFICATION
GROUNDED METHODS
INTERPRETIVE ANALYSIS QUANTITATIVE ASSESSMENT
social assessment
longitudinal methods
cladistic taxonomy
natural experiment
value chain analysis
monitoring
multi-sited ethnography
scientometrics
surveillance data mining
econometrics
case studies
network analysis
narrative analysis
historiography
comprehensive archive research
historical method
discourse analysis
process tracing
situational analysis
studying up
systems theory`
soft systems theory`
information theory
meta analysis
life cycle approaches
optimisation modelling
Bayesian models
probabilistic risk analysis
uncertainty analysis
APPRECIATIVE APPROACHES
MIXED THEORY
EVALUATIVE TECHNIQUESenvironmental
assessmentevaluation methods
statistical methods
spatial analysis
critical pedagogy
technology assessment
panel studies
social indicators
technometricscontent analysis
social experiment
randomised control trials
remote sensing
agent-based modelling
needs analysis
road-mapping
ethno-methodology
action research
counter-factuals
co-word analysis
RESEARCH METHODS
innovation histories
evidentiary presumptions
systems histories`
precautionary appraisal
critical accounting
influence mapping
post-normal science
resilience analysis
alternatives assessment
critical triangulation
cross-frame analysis
imaginaries analysis
arts catalyst
capabilities assessment
sensitivity analysis
key: analytic / interactive method
costbenefit analysis
ecological footprint
integrated assessment
largely qualitative mostly quantitative EPISTEMIC CULTURE
explicitly mixed
STYL
E O
F EN
QU
IRY
dedu
ctiv
e st
yle
abdu
ctiv
e st
yle
indu
ctiv
e st
yle QUALITATIVE APPREHENSION HYBRID APPRAISAL EMPIRICAL QUANTIFICATION
GROUNDED METHODS
INTERPRETIVE ANALYSIS QUANTITATIVE ASSESSMENT
social assessment
extended foresight longitudinal
methods
cladistic taxonomy
natural experiment
value chain analysis
monitoring
multi-sited ethnography
scientometrics
surveillance data mining
econometrics
delphi analysis
photo-elicitation
case studies
network analysis
narrative analysis
historiography
comprehensive archive research
historical method
discourse analysis
process tracing
situational analysis
studying up
systems theory`
soft systems theory`
information theory
meta analysis
life cycle approaches
optimisation modelling
Bayesian models
probabilistic risk analysis
uncertainty analysis
APPRECIATIVE APPROACHES
MIXED THEORY
EVALUATIVE TECHNIQUESenvironmental
assessmentevaluation methods
statistical methods
spatial analysis
opinion surveys
attitudinal scaling
structured deliberation
dissensus groups
citizen science
crowd sourcing
scenario workshops
futures visioning
performative approaches
focus groups
semi-structured interviews
critical pedagogy
decision analysis
contingent valuation
multicriteria mapping
Q methodgaming
techniques
open space
technology assessment
constructive technology assessment
interactive diagrams
panel studies
social indicators
technometricscontent analysis
social experiment
experimental economics
randomised control trials
remote sensing
interactive visualisation
interactive modelling
agent-based modelling
paricipatory backcasting
needs analysis
road-mapping
ethno-methodology
situational activism
action research
biography
counter-factuals
co-word analysis
participatory theatre
RESEARCH METHODS
social multicriteria analysis
stakeholder negotiation
innovation histories
concurrent evidence
evidentiary presumptions
systems histories`
precautionary appraisal
critical accounting
do-it-yourself juries
influence mapping
post-normal science
resilience analysis
alternatives assessment
critical triangulation
cross-frame analysis
power tools
imaginaries analysis
deliberative polling
diversity mapping
co-operative research
collaborative design
arts catalyst
community art
participatory rural appraisal
study groups
capabilities assessment
planning cells
sensitivity analysis
key: analytic / interactive method
costbenefit analysis
ecological footprint
INPUTS
(aspects taken into account within practice of research or appraisal)
problems, options, pros / cons, issues, uncertainties, perspectives
INPUTS
(things that are taken into account)
Pro
blems, options, pros / cons, issues, uncertainties, perspectives
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
OUTPUTS
(aspects that are conveyed outwards into wider discourse)
’Plural conditional’ conclusions…
… if X then A … if Y then B …
Rigour in Nexus-Focused Methodologies
Combining reflexivity about framing and reflection about error in informing policy and politics
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
decision analysis
INPU
TS
OUTPUTS
sustainability
safety
impacts
Rigour in Nexus-Focused Methodologies
Combining reflexivity about framing and reflection about error in informing policy and politics
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
citizen’s juries
INPU
TS
OUTPUTS
Rigour in Nexus-Focused Methodologies
Combining reflexivity about framing and reflection about error in informing policy and politics
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
citizen’s juries
participatory appraisal
q-method
scientometric mapping
open space
multicriteria mapping
extended foresightcitizen’s juries
decision analysis
stakeholder negotiation
participatory sensitivity analysis
cost-benefit analysis
risk assessment
interactive modelling
structured interviews
participant observation
multi-site ethnographic-
methods
citizen’s juries
consensus conference
open hearings
concurrent evidence
critical narratives
intervention futures
PIPA
plural photovoice
system histories
innovation histories
INPU
TS
OUTPUTS
Rigour in Nexus-Focused Methodologies
Combining reflexivity about framing and reflection about error in informing policy and politics
spot-the-narrative
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
citizen’s juries
decision analysis
participatory rural appraisal
stakeholder negotiation
q-method
sensitivity analysis
deliberative mapping
scientometric mapping
open space
cost-benefit analysis
risk assessment
interactive modelling
structured interviews
narrative-based participant observation
multi-site ethnographic-
methods
citizen’s juries
consensus conference
open hearings
dissenting opinions
multi-criteria mapping
extended foresight
stakeholder negotiation
cost-benefit analysis
risk assessment
INPU
TS
OUTPUTS
Power Closes Down Nexus Research and AppraisalPressure to command authority, foster trust, secure acceptance, manage blame
expert / analytic
participatory / deliberative
Challenges for Research and Appraisal
Reflexively means actively balancing bias in conventional appraisal
narrow
broad
closing down opening up
participatory appraisal
q-method open space
multicriteria mapping
participatory sensitivity
analysis
intervention futures PIPA
plural photovoice
system histories
innovation histories
INPU
TS
OUTPUTS
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
Power Closes Down Research and Appraisal
A responsibility for neutrality means independent innovation research and policy appraisal should deliberately counter pressures for closure
MONODISCIPLINARITY
eg: Q-method comes out of social psychology
INPU
TS
OUTPUTS
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
Power Closes Down Research and Appraisal
A responsibility for neutrality means independent innovation research and policy appraisal should deliberately counter pressures for closure
MONODISCIPLINARITY - enquiry is structured by community practices - institutionalised as “normal science” within paradigm - authority through self-confident coherence
eg: in Q-method analysis is subject to particular assumptions and conventions
INPU
TS
OUTPUTS
MONODISCIPLINARITY - enquiry is structured by community practices - institutionalised as “normal science” within paradigm - authority through self-confident coherence
Participatory methods can also be monodisciplinary
eg: participatory panel follows particular rules
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
Power Closes Down Research and Appraisal
A responsibility for neutrality means independent innovation research and policy appraisal should deliberately counter pressures for closure
INPU
TS
OUTPUTS
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
Power Closes Down Research and Appraisal
A responsibility for neutrality means independent innovation research and policy appraisal should deliberately counter pressures for closure
CROSS-DISCIPLINARITY
- task- and context-specific - allows cross-critique and triangulation - authority through juxtaposition
eg:participatory panel providescomplement to discourse analysis
INPU
TS
OUTPUTS
MULTIDISCIPLINARITY
- enquiry structured hierarchically - impressive scope, reassuring applicability - authority through integration
eg:integrated assessmentglobal panels
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
Power Closes Down Research and Appraisal
A responsibility for neutrality means independent innovation research and policy appraisal should deliberately counter pressures for closure
INPU
TS
OUTPUTS
INTERDISCIPLINARITY
- enquiry structured by collaboration - attention targeted on focal problem - authority thro’ fidelity in addressing complexity
eg: collaborative networksmultiple partnerships
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
Power Closes Down Research and Appraisal
A responsibility for neutrality means independent innovation research and policy appraisal should deliberately counter pressures for closure
INPU
TS
OUTPUTS
TRANSDISCIPLINARITY
- enquiry reflects wider extant framings - flexible to divergent contexts - authority thro’ salience and legitimacy
eg:co-designed with stakeholders
narrow
broad
closing down opening up
expert / analytic
participatory / deliberative
Power Closes Down Research and Appraisal
A responsibility for neutrality means independent innovation research and policy appraisal should deliberately counter pressures for closure
INPU
TS
OUTPUTS
Mapping Perspectives for Deliberation Multicriteria Mapping ‘opens up’ politics and power in expertise
Analysis of 12 UK government GM advisors (2001)
Mapping Perspectives for Deliberation Multicriteria Mapping ‘opens up’ politics and power in expertise
Analysis of 12 UK government GM advisors (2001)
organicslow input intensive
GM 1GM 2GM 3
organicslow input intensive
GM 1GM 2GM 3
UK Governmentecology chair
organicslow input intensive
GM 1GM 2GM 3
organicslow input intensive
GM 1GM 2GM 3
UK Governmentsafety chair
GM industry research executive
Green NGO scientist
Acknowledging assumptions, values, uncertainties ‘plural & conditional’ approach is rigorous & democratic
… if A, then x; … if B, then y
Multicriteria Mapping ‘opens up’ politics and power in expertise
Mapping Perspectives for Deliberation