ids impact innovation and learning workshop march 2013: day 1, panel session 1 richard...
TRANSCRIPT
IDS WORKSHOP Impact, Innovation, Learning
Panel Session
Values, Learning, and Systems
A framework for critical rigour
in impact evaluations
Richard Hummelbrunner
ÖAR Regionalberatung Graz, Austria
Martin Reynolds
Senior Lecturer, The Open University, UK
Bob Williams
Independent Consultant, Wellington NZ
CORE STATEMENT
…. evaluation asks and answers questions about the quality, value, and/or importance of things ….
If we’re not doing that, we’re not actually doing evaluation.
Jane Davidson: Genuine Evaluation March 2013
The System Field
Different
systems
traditions which
have shaped
contemporary
systems
practice
Some Streams of Systemic Thought (Draft update — May 2001)
* * *Originated in 1996 by Dr. Eric Schwarz, Neuchâtel, Switzerland.
Extended in 1998, including items from the The Story of Philosophy by Will Durant (1933).Elaborated in 2000-2001 from many sources for the International Institute for General Systems Studies.
Currently a research project of the IIGSS.- - -
This rendition is the property of the International Institute for General Systems Studies.
All Rights Reserved.
Errors and omissions in this chart are solely attributable to the IIGSS.
Anti-chaos &
Adaptation
Stuart Kauffman
Criticality
Per Bak
Self-organization
Dissipative
Systems
Ilya Prigogine
Complex Dynamical
Systems
Ralph Abraham
Chaos Theory
M.J. Feigenbaum
Nonlinear
Systems
Systems
Biology
Robert Rosen
Ecodynamics
Kenneth Boulding
General
Systems
Theory
Majilo Mesarovic
Dynamic Programming
Richard Bellman
Instrumental
Pragmatism
John Dewey
Psychological
Pragmatism
William James
Pragmatic Semiotics
Charles S. Peirce
Whole Systems
Design
Harold Nelson
General Systems
Philosophy
Ervin Laszlo
Dialectical
Idealism
Georg Hegel
Evolutionary
Philosophy
Herbert Spencer
Neuchatel
Evolutionary
Model
Eric Schwarz
Natural Evolution
Charles Darwin1809-1881
Cybernetics
Norbert Wiener
Information
Theory
Claude Shannon
Classical
Thermodynamics
Clausius, Boltzmann
Semantic Theory
Donald MacKay
Informatics
(Computer Science
& Engineering)
Mathematics of
Feedback &
Electrodynamics
James Clerk-Maxwell
Feedback governor
James Watt
Sciences
of the
Artificial
Herbert Simon
Social Systems
Talcott Parsons
Sociological
Systems
Walter Buckley
Social Entropy Theory
Kenneth D. Bailey
Autopoietic Social
Systems
Niklas Luhmann
Management Cybernetics
Peter Senge
Appreciative
Systems
Gregory Bateson
Social Ecology
Eric Trist, Fred Emery
Family
Systems
Therapy
Watzlewick
Systems Ecology
Howard T. Odum
Programmable Loom
Joseph-Marie Jacquard
Analytical Engine
Charles Babbage
Theory of Computation
Alan M. Turing
Rational Philosophy
Aristotle384-322BC
Symbolic Logic
George Boole
Symbolic Logic
Augustus de Morgan
Applied
Mathematics
John von Neumann
Artificial
Intelligence
Marvin Minsky
Cellular
AutomataArtificial
Neural Nets
Designing Systems
Bela H. Banathy
Teleonics
Gyuri Jaros
Critical Systems
Michael C. Jackson,Robert Flood,Werner Ulrich
Total Systems
Intervention
Robert Flood
Multi-Methodology
John Mingers
Metamodeling
John Van Gigch
Self-organized
Economics
W. Brian Arthur
Infodynamics
Stanley Salthe
Conventionalism
Henri Poincaré
Fractal Systems
Benoit Mandelbrot
Artificial Life
Chris Langton
Mathematical
Social Science
Anatol Rapoport
Chaotic Systems
Edward Lorenz
Complex Systems
(Santa Fe Institute)
Homeostatic Systems
Walter Cannon
Queuing
Theory
Holism
Jan Smuts
Tektology
A.A. Bogdanov
Knowledge
Sciences
Brian Gaines
Engineering
Archimedes
Systems
Development
John Warfield
Linkage
Propositions
Len Troncale
Idealistic Philosophy
Plato
Systemic Development
Richard Bawden
Synergetics II
R. Buckminster Fuller
Rheomodal
Systems
David Bohm
Cybernetic Semiosis
Douglas R. Hofstadter
Transformational
Linguistics
Naom Chomsky
General Semantics
Alfred Korzybski,
Systems Trends
Kenneth E.F. Watt
Geophysiology (Gaia)
James E. Lovelock
Hierarchy Theory
Howard H. Pattee
Developmental
Structuralism
Jean Piaget
Co-creative Process
Hector Sabelli
Holonomic Systems
Ken Wilber
Communicative
Action
Jürgen Habermas
Complex Evolutionary
Systems
Peter M. Allen
Knotted Systems
Louis Kauffman
Interpretive Systemology
Ramses Fuenmayor,Hernan Lopez-Garay
Informational
Microdynamics
Vladimir Lerner
Social Semiotics
Floyd Merrell
Socio-Cybernetics
R. Felix Geyer
Pansystems
Wu Xuemou
Critical Ontology
Michel Foucault
Metasystem Transition
Turchin, Heylighen, Joslyn
Systemic Selfness
Paul Ryan
Homeorheotic
Systems
William Irwin Thompson
Systems Semiotics
Luis Rocha,Howard Pattee
Epistemology of Science
Sir Arthur Eddington
Time
Dilation
Lorentz
Topology of
Meaning
R. Ian Flett
Philosophy of Physics
John Archibald Wheeler
Mathematical Topology
Catastrophe
Theory
René Thom
Geometrodynamics
Albert Einstein
Quantum Systems
Planck, Bohr, de Broglie, Dirac,
Schrödinger
Semantic Pragmatism
Ludwig Wittgenstein
Geometry of Meaning
& Reflexive Universe
Arthur M. Young
Abstraction of
Physical Laws
Galileo Galilei
Observational
Astronomy
Tycho Brahe
Heliocentric Astronomy
Nicolaus Copernicus
Socio-Technical
Systems
Harold Linstone
Software
Theory
Social
Psychology
Abraham Maslow
Systems
Anthropology
Margaret Mead
Mathematical
Astronomy
J. Kepler
Structuralism
Claude Levi-Strauss
Programming
Ada Lovelace
Engineering Design
Leonardo da Vinci
Human Science
Giambattista Vico
Geological
Superorganism
James Hutton
Biosphere
Eduard Suess
Earth as
Organism
Yevgraf Korolenko
Earth as Organism
Vladimir Vernadsky
Species Cybernetics
Lotka & Voltarra
Holonics
Jeffrey Stamps
Politics of Benefit
Cicero
Electronic Digital
Prototype
Atanasoff & Berry
Electronic Digital
Computing
Eckert & Mauchly
Operating Systems
Brooks et al
Operating Languages
Kenneth Iverson et al
Endosymbiosis
Lynn Margulis
Fuzzy
Systems
Lofti Zadeh
Fuzzy Logic
Bart Kosko
Calculus of
Variations
Dialectical
Materialism
Karl Marx
Paradigmatic
Revolutions
Thomas Kuhn
Periodic
Table
Dmitry Mendeleev
Systematic Medicine
Hippocrates
Medicine
Celsus
Clinical Medicine
Thomas Sydenham
Conservation of matter
Antoine Lavoisier
Physical Chemistry
Robert Boyle
Antibacterials
Alexander Fleming
Electromagnetic
Propagations
Heinrich Hertz
Antisepsis
Lister
Clinical
Psychiatry
Benjamin Rush
Conservation
of Energy
Joule
Intrinsic
Uncertainty
Heisenberg
Superstrings
Brian Greene, et al
Evolution
Alfred Wallace
Pendulum Clock &
Wave Theory of Light
Christiaan Huygens
Determinism
Laplace
Scientific
Philosophy
Jacques Monod
Mathematical
Nominalism
Gottfried von Leibniz
Immaterial
Empiricism
George Berkeley
Ecology
C.S. Holling
Bio-cybernetics
Manfried Eigen
Transcendentalism
Ralph Waldo Emerson
Natural Philosophy
Nicholas von Cusa
Political Hegemony
Frederick the Great
Semiotic
Umwelt
Jakob von
Uexküll
Bio-semiosis
Jesper
Hoffmeyer
Comparative Linguistics
Benjamin Lee Whorf
Atomisitc
Linguistics
Antoine Fabre d'Olivet
Symbolic Logic
Willard Van Orman Quine
Operating
hardware
Blauw et al
Linear
Programming
Optimality
Theory
Synergy
Peter A. Corning
Appreciative Systems
William E. Smith
Holonics
Arthur Koestler
Mathematical Philosophy
Pythagorasc.580-500BC
Geometry
Euclid
Logical Positivism
Rudolf Carnap
Process
Henri Bergson
Meta-Skepicial
Empiricism
David Hume
Political Pragmatism
John Locke
Political
Romanticism
J.J. Rousseau
Critical Transcendental
Idealism
Immanuel Kant
Fluxation& Unified Opposites
Heraclitusc. 540-480 B.C.
Cynic
Philosophy
AntisthenesCynicism
Diogenes
Dialectic
Stoicism
Zeno of Citium
Religious Stoicism
Epictetus
Stoicism
Marcus Aurelius
Cosmological Unity
Giordano Bruno
Philosophy of the WholeBaruch Spinoza
Nature Philosophy
von Schelling
Will & Idea in Nature
Arthur Schopenhauer
PerspectivismFriedrich Nietzsche
Noosphere
Teilhardde Jardin
Decision
Theory
Geometric
ProgrammingUtility
Theory
Laws of Form
George Spencer-Brown
Blown Up Systems
Shoucheng OuYang
Structured
Software
Yourdon, Constantine, DeMarco,
Critical Theory of
Society
Max Horkheimer, Theodor Adorno
Ecological
Cybernetics
Garrett Hardin
I Ching
2000-1000 B.C.
Social Cybernetics
Stuart Umpleby
Eco-pathology
Rachel Carson
Sructuralist
Mathematics
Bourbaki
KEY: whit e general syst ems
red cybernet ics
black physical sciences
blue mat hemat ics
magent a comput ers & informat ics
green biology & medicine
yellow symbolic syst ems
orange social syst ems
olive ecology
gray philosophy
cyan syst ems analysis
purple engineering
General
Systemology
D.H. McNeil
Anthropological
Linguistics
Edward Sapir
International Encyclopedia of
Systems & Cybernetics
Charles François1997
Mathematical
Logic
Bertrand Russell
General Systems Theory
Yi Lin
Classical Mathematics
Development
of Mentation
Walter Lowen
Punctuated Evolution
Stephen J. Gould
Communicational Cybernetics
Klaus Krippendorff
Atomism
Democritus
Sophistic
Philosophy
Socrates
Organizing relations
Needham, Woodger
Rationalistic
Dualism
René Descartes
Analysis of
Complexification
Conrad H. Waddington
Radical
Constructivism
Ernst von Glasersfeld
Autopoietic
Systems
Rod Swenson
Unity & Stasis
Parmenides
Atomism
Leucippus
Dialectic
Zeno of Elea
Naturalism
Thales
Reciprocities
Anaximander
Scientific
Positivism
Francis Bacon
Scientific Empiricism
Roger Bacon
Scientific Method
Robert Grosseteste
Pan-ecology
John Muir
Systems
Philosophy
Mario Bunge
Modeling
J.L. le Moigne
Inheritance
Gregor Mendel
Genomics
Craig Venter,Francis Collins
Systemics of
Benefit
Warren Ziegler
Eco-futurism
Hazel Henderson
Critical
Phenomenology
Edmund Husserl
Empirical
Sensationalism
Etienne Condillac
Evolutionary
Organicism
Saint-Simon
Empirical
Positivism
Ernst Mach
Positivism &
Sociology
Auguste Comte
Positivism
Lucretius
Positivism
Epicurus
Positivism
Aristapippas
Utilitarianism
Jeremy Bentham
Empirical
Utilitarianism
J.S. Mill
Transcendental
Idealism
Johann Fichte
Psychological
Determinism
Thomas Hobbes
Determinism
Voltaire
Progressivism
Condorcet
Intentionality
Franz Brentano
Existentialism
Martin Heidegger
Demographic
Cybernetics
Thomas Malthus
Political Economy
Adam Smith
Modern
Economics
John Maynard Keynes
Economics
Paul Samuelson
Simulation
Problematiques
& Planning
Hasan Ozbekhan
Autopoietic
Systems
Humberto Maturana, Francisco
Varela
Harmony of Opposites
Lao Tzu
c. 570-490 B.C.
Statistics
Karl Pearson
Pagan Celts
Philosophy of
Science
Fritjof Capra
Metaphysics of
Science
Gary Zukav
Economic
Cybernetics
David Ricardo
Information
Theory
L. Brillouin
Linguistic
Mathematics
Hekki Heiskanon
Relativism
Postmodernism
Evolutionary
Positivism
Richard Dawkins
Deconstructionism
Jacques Derrida
Relativity of
Rationalites
Feyerabend
Structuralistic
Semiology
Ferdinand de Saussure
Sociology
Emil Durkheim
Narratology
Roland Barthes
Language &
Symbolism
Comparative
Languages
William Jones
Literary Semiotics
Umberto Eco
Logical Positivism
A.J. Ayer
Scholasticical
Empiricism
Thomas Aquinas
Personalism
Renouvier
Rationalism
Avicenna
Rationalism
Averroës
Psychoanalysis
Sigmund Freud
Analytic
Psychology
Carl Jung
Scholasticical
Sciences
Albertus Magnus
Scientific Philosophy
Paul C.W. DaviesPhilosophy of Physics
& Mathematics
F. David Peat
Electromagnetism
Ampere, Kirchhoff, Faraday et al
Electrostatics
Benjamin Franklin
Magnetism
Gauss
Biostructures
(chirality)
Louis Pasteur
Medicine
Moses Maimonides
Scholastical Logic,
Dialectic & Linguistics
Peter Abelard
Scholasticism
Saint Anselm
Noetic Systems
Walter J. Ong
Semiotic Catholicity
Randolph F. Lumpp
Media
Semiotics
H. Marshall McLuhan
Skepticism
Michel Montaigne
Materialism
Denis Diderot
Management
Science
Philosophy of Regularity
Philosophy of Irregularity
River Map & Luo Book
Sociality
& Self
Confucius
YinYang
Fu Xi
Divination
Formal
Regularity
Zhu Xi
Tessellations
Roger Penrose
General Systems
Evolution
Erich Jantsch
Ecological
Demographics
Paul Ehrlich
Stabile Systems
Claude Bernard
Paradoxical
Imagery
M.C. Escher
Anatomical
Medicine
Galen
Medicine
Avenzoar
Medicine
Taddeo Alderotti
Medicine
Guy de Chauliac
Anatomical Medicine
Anreas Vesalius
Chemical Medicine
Aureolus Paracelsus
Microbials
von Leeuwenhoek
Circulation
William Harvey
Modern Theoretical Physics
Classical Physics
Critical Physics
Ronald W. MosesCosmological
Physics
Stephen Hawking
Environmentalism
David Brower
Dialectical
Naturalism
Friedrich Engels
Grand Unified Treories
Steven Weinberg
Fuzzy Systemics
Vladimir Dimitrov
General System Theory
Ludwig von BertalanffyOperational Research
Russell L. Ackoff,Stafford Beer
First Order Cybernetics
Sciento-politics
Ian Mitroff
Political
Systemics
Anthony Wilden
Military
Hegemony
Alexander the Great
Scientific
Intuitionism
Michael Polanyi
Self-Reference
& Autonomy
Control Theory
Operations Research
General Systems
Purposeful Systems
Russell L. Ackoff
Philosophy
of Systems
Thomas Cowan
Experimentalism
E.A. Singer
Inquiring Systems
C. West Churchman
General Systems
Thinking
Gerald M. Weinberg
Management Cybernetics
Barry Clemson
Systems Cybernetics
W. Ross Ashby
Organicism & Process
Alfred North Whitehead
General Systems
Modeling
George Klir
Perceptual
Control
Theory
William T. Powers
Living Systems
James Grier Miller
Control
System
Engineering
Systems Dynamics
Jay W. Forrester,Donella Meadows
DNA
Crick, Watson
Immunization
Jonas Salk
Directive
Correlation
Sommerhoff
Conversation
Theory
Gordon Pask
Politics of Cynicism
Niccolo Machiavelli
Mechanistic
Physics
Isaac Newton
Critical
Epistemology
Karl Popper
Linear &
Nonlinear
Mathematics
John Casti
Hyper-
mathematics
Charles
Muses
Soft Systems
Methodology
Peter Checkland
Rational
Mechanics
Approximation-
Estimation Theory
Grey Systems
Sifeng Liu, Yi Lin
General Tropodynamics
Soucheng OuYang, Yi Lin
Algebra
Aryabhata
Algebra
Brahmagupta
Algebra
Bhaskara
Algebra
Omar Khayyam
Algebra
Fibonacci
Rotational
Invariance
P.W. Bridgeman
Indian
Philosophy
Scientific Management
Frederick W. Taylor
Recyclical Universe
Hindu Mythology
Competitive
Evolution
Thomas Huxley
Systems &
Procedures
Heroic Legends
Homer
Rosetta Stone
Egypt
Applied
General Semantics
S.I. Hayakawa
Cyber-semiotics
Søren Brier
General Semiotics
Thomas Sebeok
Synergetics
Hermann Haken
Comparative
Anthropology
Franz Boas
Systems Engineering
Harold Chestnut,Andrew P. Sage
Systems Analysis
Arthur D. Hall,Van Court Hare
Second Order
Cybernetics
Heinz von Foerster
Topology of
Cybernetics
Warren McCulloch
Viable Systems
Stafford Beer
Holographic
Systems
Karl Pribram
Constitutional
Pragmatism
James Madison
Philosophy of
Systems
Archie BahmBehavioral
Psychology
B.F. Skinner
Positivism
Josef Popper-Lynkeus
Mathematical Logic
Gottlob Frege
Measure Theory
Giuseppe Peano
Projective
Geometry
Georg Cantor
Imperial Hegemony
Roman Empire
Genetics
William Bateson
Social Cybernetics
Geoffrey Vickers
Democracy
Cleisthenes
Cuneiform
Sumerians
Hieroglyphics
Egyptians
Alphabet
Phoenicians
Regularized
Language
Indo-Europeans
Democratic Idealism
Thomas Jefferson
Democratic Development
Pericles
Astronomy
Babylon
Classical Biology
Wholistic Biology
Barry Commoner
Formalized
Logic
Semio-Physics
René Thom
Group Theory
Felix Klein
Linear
Analysis
J. Fourier
Algorithmic
Complexity
Theory
G. Chaitin
Ideographics
Chinese
Pictographics
Aboriginals
Formalism
David Hilbert
Linear
Analysis
Dirichlet
Social
Systems
Sciences
Field
Theory
Evariste Galois
Algebraic
Functions
Bernhard Riemann
Analytic Geometry
Pierre de Fermat
Analytic Functions
Leonhard Euler
Analytic Mechanics
Joseph-Louis Lagrange
Imperial
Hegemony
Napoleon
Probability Theory
Blaise Pascal
Algebra
Diophantus
Social Psychology
Willis F. Overton
Mathematics of
Concentric Spheres
Callippus
Astronomy
Eudoxus
Geometry
Theudius
Law
Solon
Conic Sections
Apollonius
Atomic
Weights
John Dalton
Inorganic
Chemistry
Organic
Chemistry
Liebig
Bio-Taxonomy
Carolus Linnaeus
Gradual Geological
Evolution
Charles Lyell
Geological
Tectonics
Symmetry
A. Zee
Manual
Transcriptions
Monastic Scribes
Printing
Johannes Gutenberg
Observational
Astronomy
Hipparchus
Geocentric
Astronomy
Ptolemy
Polymath Philosophy
Nicholas Oresme
Quaternions
William Hamilton
Internet
ARPA
Telecommunications
& Data Bases
James Martin
Apocalyptic
Dualism
Zoroasterc.628-551BC
Numerical
Analysis
Hypervisors
Emulation
Collateration
& Hypertext
Ted Nelson
World Wide Web
Tim Berners-Lee
Systemic Perspectivism
Experimental
Psychology
Wilhelm Wundt
Psychological
Modeling
Clark Hull
Clinical
Psychology
Techno-
Psychology
Psycho-
Biology
Psycho-
Physics
Morphogenesis
D'Arcy Thompson
Geo-Impact
Evolution
Eugene Shoemaker
Mathematical
Biology
A.J. Lotka
Eigenvalues &
Superposition
Daniel Bernouli
Genetic
Algorithms
Sturm-
Louiville
Theory
Systems
Analysis
Perturbation
Theory
Generalized
Uncertainty
J. Ford
Documentary
& Narrative
Phonemes
Roman Jakobson
Hieroglyphics
Mayans
Social
Anthropology
Bronislaw Malinowsky
Illuminationism
Saint Augustine
Rational
Voluntarism
John Duns Scotus
Nominalism
William of Ockham
Logical Nominalism
Jean Buridan
Interpretive
Physics
B.J. West
Mechanical Calculator
Wilhelm Schickard
Arithmometer
Charles de Colmar
Tabulating Machine
Herman Hollerith
Rapid
Miniaturized
Computing
Nanotechnology
Encoding
Schemes
R.W. Hamming
Philosophy of
Nature & Science
Carl von Weizsäcker
Bifurcated
Cultures
C.P. Snow
Evolutionary
Systematization
Pierre Duhem
Pattern
Recognition
Psycho-
Metrics
Ecological
Evolution
Ernst Haeckel
Molecular
Genetics
Empirical
Metaphysics
George Henry Lewes
Evolutionary
Philosophy
Richard L. Coren
Comparative
Psychology
C. Lloyd Morgan
Emergent Process
Samuel Alexander
Linear Algebra
Benjamin Peirce
Algebra of Logic
Ernst Schröder
Associative Algebras
William Clifford
Statistical
Biology
Karl Pearson
Modern
Statistics
Jerzy Neyman
Statistical
Mechanics
J. Willard Gibbs
Model Theory
Bernhard Bolzano
Model Theory
Alfred Tarski
Model Theory
Michael Morley
Metalogic
Alonzo Church Metalogic
Kurt Gödel
Intuitionism
L.E.J. Brouwer
Category
Theory
Saunders Mac Lane
Double
Refraction
Augustin Fresnel
Metric
Units
Astronomical
Measurement
Pierre Mechain
Sociological
Phenomenology
Max Scheler
Philosophy of
History
Wilhelm Dilthey
Renaissance
Platonism
Giovanni Pico
Number Theory
Marin Mersenne
Polymath Physics
Robert Hooke
Industrial Sociology
G. Elton Mayo
Philology, Linguistics
& Grammar
Ethnolinguistics
Wilhelm von Humboldt
Ethnolinguistics
Johann Herder
Etymology
Jacob Grimm
Tagmemics
Kenneth L. Pike
Stratificational
Grammar
Sydney Lamb
Glossematics
Louis Hjelmslev
Morphology
Historicism
Benedetto Croce
Conceptualism
Ernst Cassirer
Tropology
Practical Medicine
Egyptians
Colonial
Hegemony
British Empire
Cultural
Hegemony
USA
Moral
Philosophy
Friedrich Schiller
Organicism
Edward Stuart Russell
Philosophy
of Biology
Morton O. Beckner
Number Theory
G.H. Hardy
Philosophy
Archelaus
Philosophy
Anaxagoras
Cognitive
Psychology
Aaron Beck
Structural Linguistics
N.S. Trubetzkoy
Realism
G.E. Moore
Analytical
Philosophy
Gilbert Ryle
Sophism
Protagoras
Game
Theory
Automata
Theory
Software
Measurement
Capers Jones
Binary Numbers
China
Meta-languages &
Linguistics
U. of Nalanda, India
Algorithms
Donald Knuth
Structuralist
Psychology
Edward Titchener
Gestalt
Psychology
Max Wertheimer
New Criticism
John Crowe Ransom
Elementalism
Empedocles
Developmental
Psychology
Cognitive
Science
Allen Newell
Mechanistic Materialism
Julien La Mettrie
Computationism
Jerry A. Fodor
Literary
Criticism
I.A. Richards Standardized
Linguistics
C.K. Ogden
Critical Philology
Desiderius Erasmus
Humanism
Petrarch
Cognitive
Linguistics
George Lakoff & R.W. Langacker
Intensional
Logic
Montague
Linguistic Logic
Jerrold J. Katz
Philosophy
of Art and
Language
Susanne K. Langer
Problem-
Sovling
Propositional Logic
Chrysippus
Religious Stoicism
Cleanthes
Diogenes of Babylon
Roman Stoicism
Panaetius
Roman Stoicism
Poseidonius
Roman Stoicism
Seneca
Tectonic
Engineering
John Roebling
Industrial
Design
Coherentism
Otto Neurath
PERT
Delphi
Conditional
Probability
Thomas Bayes
Enlightenment
Jean d'Alembert
Ecology
Ethology
Konrad Lorenz
Objective
Information
T. Stonier
Biological
Evolution
Jean Lamarck
Vorticles
William Thomson
Sociobiology
Edward O. Wilson
Systematics
Ernst Mayr
Interpersonal
Psychology
R.D. Laing
Cybernetic
Epistemology
Ranulph
Glanville
Mathematical
Economics
Antoine Cournot
Idealistic Liberalism
William Godwin
Mathematical
Psychology
R.D. Luce
Neo-Platonism
Plotinus
Social
Psychology
Kurt Lewin
Gestalt
Psychology
Wolfgang Köhler
Mathematical
Economics
Vilfredo Pareto
Psycholinguistics
Sociolinguistics
Computational
Linguistics
Linguistic
Psychoanalysis
Jacques Lacan
Group
Dynamic
Sociometrics
Political
Sociology
Max Weber
Econometrics
Philosophy of
Law
Hans Kelsen
Eugenics
Francis
Primate
Anthropology
Jane Goodall
Primate
Linguistics
Linguistic Science
Leonard Bloomfield
Phonology
Physics As
Metaphor
Roger S. Jones
Applied
Transcendentalism
Henry David Thoreau
Instrumental
Rationality
Peter Ramus
Methodized
Encyclopedias
Johann Heinrich
Alsted
Conscientization
Paolo Freire
Robotics
Biodiversity
Urban
Ecology
Louis Wirth
Biotechnology
Genetic
Engineering
Green
Revolution
Norman
Borlaug
Heredity
P-L. de Maupertuis
Psychoanalysis
Josef Breuer
Core System Concepts
Understanding interrelationships associated with a
situation;
Engaging with contrasting perspectives regarding a
situation, and
Reflecting on boundaries of such representations and
interactions
INTERRELATIONSHIPS – Dynamics, Context, Complexity
• What is the structure of interrelationships within the situation (how are
components of intervention and/or effects connected)?
• What is the nature of the interrelationships (e.g. strong/weak, fast/slow,
direct/indirect, linear/non-linear)?
• What are the patterns that emerge from the interrelationships between
intervention and effects over time (e.g. simple, complicated, complex)?
• How context sensitive are these patterns (different effects at different times
or in different contexts)?
• How will this affect the ways in which effects can be associated with the
intervention? Understand what really works in what circumstances?
INTERRELATIONSHIPS – Relevance to Impact Evaluation
Emergence
Unintended & Unanticpated Consequences
Need For Keeping Measures Adaptable Rather Than Fixed
PERSPECTIVES – Stakeholders, Stakes, Framing
PERSPECTIVES– Relevance to Impact Evaluation
Untangling map from territory
Assessing why things worked out the way they did
Challenging idea of single “system”
BOUNDARIES – values, power, knowledge, legitimacy
BOUNDARIES– Relevance to Impact Evaluation
Broadening the range of values (utility, rights, virtues)
Identifying the values underpinning the intervention
Identifying the values underpinning the impact evaluation
Types of Learning
Effects Intervention Values,
assumptions
Rules,
knowledge
Single loop learning
Double loop learning
Triple loop learning
Adaptive actions
Underlying causes and consequences
Mechanisms for learning, coping with problems and changes
Types of Learning and Systems Concepts
Perspectives Boundaries
Single loop learning
Double loop learning
Triple loop learning
Interrelationships
Are we doing
things right?
Are we doing the right things?
Is rightness buttressed by mightiness (or vice versa)?
Questions purpose, goals
Power for determining purpose and goals
Intervention Effects
A systems - based framework for rigor in evaluation
Intervention
Perspectives
Boundaries
Single loop learning
Double loop learning
Triple loop learning
Interrelationships
Instrumental values
Critical /
Political
values
Intrinsic /
personal
values Effects
(ii) Partial in
serving some
stakeholder parties
including
practitioners - or
interests - better
than others
(i) Partial in
representing only a
section rather than
the whole of the
total universe of
interrelationships in
any context that
matters
All systems are partial
Systemic rigour and impact evaluation Framework 2 (triangulation): 3 systems concepts in action
1. Complex
situations or
Contexts of
change
and uncertainty
All Systems are partial in the sense of :
1. representing only a section rather than the total
universe of considerations
and
2. serving some parties - or interests - better than
others
Political space for exploring boundary judgements
Socio-economic space
for exploring
judgements of
‘fact’
Learning space for
exploring value
judgements
(i) Framing
interrelationships and
interdependencies
(ii) Framing engagement
with multiple
perspectives
3. Systems and other conceptual Tools
(iii) appreciating limits on
boundaries of interrelationships
and perspectives
Framework for justifying boundaries of:
(i) Framing interrelationships
(ii)Framing multiple perspectives
2. People
or stakeholders/
practitioners
Critical rigour and impact evaluation Framework 3 (systems thinking in practice): 3 systems concepts in action
Relevance to Impact Evaluation Emphasis of CAS and Critical
System
Inter-
relationships
Emergence and uncertainty of impacts = systemic effect from interaction
of parts rather than single causality
… whole is more than the sum of it’s parts
CAS investigates the territory
Critical systems investigates the
implications of different maps of the
territory
Multiple
Perspectives
Complexity of ‘impacts’ arise from interaction between different
stakeholders and stakes
CAS illustrates different perspectives
predominantly from value-perspective
of utility
Critical systems explores the value
component of different perspectives
and ‘developing value’
Boundary
Judgements
Systems thinking explores single, double and triple loop learning in
terms of not only:
1. Is it being done right (efficacy, efficiency)? but also…
2. Is it right thing to do (effectiveness and ethics)? and...
3. Is rightness confused with mightiness (politics and power
relations)?
CAS largely focuses on efficacy,
efficiency and effectiveness
Critical systems focuses in additon on
effectiveness, ethics and power
Critical systems Relevance and complexity
General
• Good impact evaluation requires address to perspectives and boundaries in addition (& in relation)
to complexity of interrelationships
• How might systems concepts complement and enhance (rather than replace or merely add to)
existing evaluation tools?
Interrelationships
• Measures of success (in a system) must adapt to change in circumstances (in situations)
• In what way can systems ideas improve the rigour and relevance of impact evaluation?
Perspectives
• Purposeful (as against ‘purposive’) developmental evaluation requires acknowledgement of, and
engagement with, ‘victims’ of intervention amongst others, in generating new value
• What are the conditions under which using systems ideas in impact evaluation is feasible and
value-able?
Boundaries
• Learning in impact evaluation requires address not only to utility (who learns what when?), but
changes in rights (and values) and responsibilities (power relations) ….triple loop learning
• What are the conditions under which using systems ideas in impact evaluation might be worthwhile
(good, right and politically just)?
Challenges from a (critical) systems perspective Questions for discussion
Systems Concepts in
Evaluation. An
expert
anthology
(AEA
publication)
eds. Bob
Williams & Iraj
Imam
2007
Systems Approaches to
Managing
Change: a
practical guide
eds. Martin
Reynolds &
Sue Holwell
2010
Systems Concepts in
Action : a
practitioners
toolkit
eds. Bob
Williams &
Richard
Hummelbrunner
2010
Richard Hummelbrunner
Senior Partner
ÖAR Regionalberatung Graz
Austria
+43 316 31 88 48 - 15
http://www.oear.at
email: [email protected]
Martin Reynolds
Senior Lecturer
The Open University
UK
+44 (0)1908 654894
http://sites.google.com/site/jjntest1/Ho
me/people/martin-reynolds-1 email: [email protected]
Bob Williams
Independent Consultant
Wellington
New Zealand
(64) 21 254 8983
http://www.bobwilliams.co.nz
email: [email protected]
Contacts ...and some book publications