monitoring and evaluation of knowledge management elb
DESCRIPTION
Presentation from the IKM-Emergent group presenting work on M&E of knowledge management. Presentation given during the KMIC webinar organised by USAID.TRANSCRIPT
Monitoring and Evaluation of Knowledge Management
Simon Hearn, ODI, [email protected]
Ewen LeBorgne, IRC International Water and Sanitation Centre, [email protected]
Valerie Brown, Australia National University, [email protected]
Overview
“It is, in fact, nothing short of a miracle that the modern methods of instruction have not entirely strangled the holy curiosity of inquiry.”
- Albert Einstein
Definitions
“When I use a word, it means just what I choose it to mean — neither more nor less”
Monitoring and evaluation
• OECD definitions: – Evaluation: The systematic and objective
assessment of an on-going or completed project, programme or policy, its design, implementation and results.
– Monitoring: A continuing function that uses systematic collection of data on specified indicators to provide indications of the extent of progress and achievement of objectives (abridged)
Monitoring and evaluation
• OK, but... Any definition must recognise:– M&E as universal functions, not specialised roles– Presence of different worldviews– Validity of evidence from different knowledge
domains*– The ethical basis for the desired social change– The importance of the unexpected and the
intangible
Knowledge
• Objective and subjective• Individual and society• Facts and values• Tacit and implcit• E.g. Western scientific conception of
knowledge as ‘justified true belief’ vs African concept of Ubuntu
Development
• Often conceptualised as a service industry• Delivery of even basic services (roads,
sanitation..) requires an understanding of the social, political and economic contexts
• Thus, development is more like a knowledge industry (Powell 2006)
• But development is more than donor aid and we must recognise civic-driven change also
Challenges
Challenges in M&E of KM4D1. KM4D does not as yet have a well grounded theory2. Knowledge for development practice is still young3. KM4D goes beyond what is labeled ‘KM’4. Competing ontological and epistemological perspectives
(and related knowledge systems)5. Existing reporting frameworks are designed for a service
industry rather than a knowledge industry6. There can be no simple cause-effect relationship7. KM initiatives often lack explicit linkages to individual,
specialist, organisational or social results8. Knowledge is not static9. Lack of methods for interpreting intangibles
Signposts
1. KM ripple model
Performance improvement
Changed practices
Knowledge capital
Knowledge process-
enhancing activities
Hulsebosch et al (2009)
2. The KM Framework
Talisayon (2009)
Need a better understanding of what intangibles are
Human Capital
Structural Capital
Relationship Capital
Tangible AssetsM
otivational Factors
Cognitive Factors
Based on Talisayon (2009)
Value creation through intangibles
Need a better understanding of knowledge transitions
Nonaka and Takeuchi (1995)
SECI
Need a better understanding of how knowledge is put to use
Graham et al (2001)
Knowledge to action cycle
Need a better understanding of organisational factors affecting knowledge use
Ramalingam (2005)
The RAPID Framework for Knowledge Strategies
We need to understand the level of complexity
Snowden (2002)
Cynefin framework
Summary: a range of perspectives• Ontological: What world-views are reflected in the
initiative and how do we recognise them? • Epistemological: What are the knowledge domains
contributing to M&E and how do they relate?• Socio-political: Who has a stake in the monitoring
process and who has power? How can we monitor these interdependent relationships?
• Methodological: How to choose tools and approaches relevant to the parties and processes involved?
• Operational: How do we organise M&E activities according to each of the knowledge domains?
Your reflections?
• Do you identify with these signposts?• What signposts do you use?• How do you see these models supporting
your work?
Multiple knowledges:
M&E as multiple partners
Whole-of-community M&E
HOLISTIC SOLUTIONS
SPECIALISED ADVICE
ORGANISATIONAL SUPPORT
COMMUNITY INTERESTS
INDIVIDUAL COMMITMENT
Multiple knowledges (Brown 2008)
HOLISTIC KNOWLEDGE Focus, vision.
SPECIALISED KNOWLEDGE Environment, Health, Finance…,
STRATEGIC KNOWLEDGE
Organisational structure, aims
LOCAL KNOWLEDGE Shared community event
INDIVIDUAL KNOWLEDGE Personal lived experience
Rejected knowledges
HOLISTIC KNOWLEDGE
Airy-fairy.
SPECIALISED KNOWLEDGE Jargon
STRATEGIC KNOWLEDGE
Self-serving
LOCAL KNOWLEDGE
Anecdote
INDIVIDUAL KNOWLEDGE Biased
25
Local knowledgeLocal knowledge
Holistic knowledge Holistic knowledge
Individual knowledgeIndividual knowledge
Collective knowledge as a nested set
A collaborative systemA collaborative system
Specialist knowledgeSpecialist knowledge
Organisational knowledge
26
Port Pirie: small town with the biggest lead smelter in the world
COMMUNITY
SPECIALISTSPECIALIST
ORGANISATIONORGANISATION
HOLISTIC FOCUSHOLISTIC FOCUS
People long resigned to risk People long resigned to risk
Health Centre stays aloof Health Centre stays aloof
Mine muzzles council Mine muzzles council
INDIVIDUAL Children diagnosed with lead Children diagnosed with lead
KNOWLEDGES STRUCTURE KNOWLEDGES STRUCTURE CONDITIONSCONDITIONS
Fear for future livelihood
27
New alliances in Port Pirie
COMMUNITY COMMUNITY
SPECIALISTSSPECIALISTS
ORGANISATIONALORGANISATIONAL
HOLISTICHOLISTIC
Outrage, political actionOutrage, political action
Technical skills, advocacyTechnical skills, advocacy
Public/private goodPublic/private good
Children’s well-beingChildren’s well-being
INDIVIDUALNDIVIDUAL Parent, grandparentParent, grandparent
M&E as collective learning- multiple interests
- multiple knowledges
- collaborative action
Next steps: - The IKM-E approach- Emergent questions on the horizon
Our approach: Multi-evidence based?
Each knowledge community uses different M&E criteria, evidence bases, databases for judgments...
• Individuals (experiences)• Communities (observations)• Experts (practitioner stories)• Organisations (monitoring reports as stated)• Holistic thinkers (ideas, forecasts)
Our approach: Purposes of conducting M&E
• Financial accountability• Operational improvement• Strategic readjustment• Capacity strengthening• Contextual understanding • Deepening understanding (research)• Self-auditing• Advocacy• Sensitisation
(From I. Guijt’s PhD thesis ‘seeking surprise’)
Our approach: KM as collective learning
Describe
DesignDo
Develop
Initiative
Key to nested knowledge cultures:
Individual (Local) Community Specialised Organisational (strategic) Holistic
Our approach: critical questioning
• A series of questions at each step of the way– Overall, a sound questioning practice– And specifically, a guideline to tailor one’s
approach:• What questions to address?• Who to involve, in what function?• What tools and methods to choose?• What lessons to draw from the approach?
Emergent questions on the horizon
• How would our approach work in practice?
• Specific methods and metrics to go ‘light’
• Particularly complexity-focused approaches
• Power vs. collective?
What now?
IKM-E + KMIC = IKMEKMIC?
• Avoiding overlaps...– Connecting KMIC and IKM (blogs...)– Organising another webinar?– Identifying different models / approaches?
• Having creative leaps...– Reviewing the IKM papers?– Expanding parts of this paper? – Testing the IKM-E framework (later)?
Additional resources
• IKM-Emergent website: http://wiki.ikmemergent.net
• The giraffe, Working group 3 blog• Working paper 3: ‘Monitoring and Evaluation in
Knowledge Management for Development‘ http://su.pr/5rqp8c
• Background paper: ‘Monitoring and evaluating knowledge management strategies’ http://su.pr/28Q9Yu
Thank you!