rbm: international experience - evaluationcanada.ca

© Universalia

RBM: International Experience

Reflecting on Implementation

16/06/2009 © Universalia 2

Implementing

“At this stage, the XX has not designed, nor is it implementing, a change management strategy to support RBM. Some of the staff interviewed are skeptical about the intentions and progress of RBM, identifying it as an externally imposed agenda or as an agenda that means nothing new for the XX.”

(Organizational Review of the implementation of RBM within a Multilateral Organization 2005)

What is RBM?

• A management approach which puts a great deal of emphasis on meeting “objectives” outcomes – while at the same time controlling inputs and producing appropriate products or services

• Objectives are translated into outcomes which involve changes in people, organizations, policies...or environmental conditions

• RBM is built on Program and Change logic systems• It is a rational linear modeling and management

approach• It is used for planning, monitoring and evaluation


It can be all things...

• Face validity• Rational• “Motherhood” • Provides and image of – Accountability – you

are in charge – control• Has a technical managerial persona• Is systemic• Anyone can be a RBMer!!!! The club is

inclusive16/06/2009 © Universalia 4

Why is Implementation of RBM a Constant Challenge?


RBM History


RBM and its Evolution

• Bureaucracy – Ideal model – it is about control • Science of Management Production 1900s• 1930s – People count • 1956 MBO – Making management more rational, goal oriented and control oriented • 1962 RBM in Private Sector – Drucker• 1950s and 1960s Program Based Budgeting – private and public• 1960s – onward – various cults of efficiency management sciences – OD• USAID – 1968 Logical framework-Social program Evaluation field blossoms• 1980s – ZOPP – added participatory processes to logic models• USAID, DFID 1990’s RBM introduced more Formally• CIDA 1994-95 Evaluation Division proposes RBM Policy• Washington 1st Roundtable 2002-RBM • Marrakech 2nd Roundtable 2004-MFDR• Paris Declaration 2005 – RBM a specific characteristic of “good aid delivery”• Hanoi 3rd Roundtable 2007 MFDR reinforced• Accra: High level Forum on Aid Effectiveness 2008 MFDR continues


OBSERVATION 1: RBM IS NOT NEW EITHER IN INTENT OR IN PRACTICE –PERSISTENT CALLS FOR BETTER RESULTS SYSTEMS

RBM Internationally

• Logical Framework – LFA – Zopp – 1968• RBM Formally introduced in USAID and DFID 1990• CIDA 1994 • Global at a string of meetings starting in 2002• Paris Declaration – 2005

– Harmonizing practices– Alignment– Changing “ownership” of aid activities– Mutual accountability – Focus on Results – Millennium Development Goals

• Accra


Purpose

• To place RBM in the larger perspective of management thought

• Opportunity to reflect on my experiences about the practice of implementing

• To challenge you and your way of understanding what RBM is about


My Involvement

• 1960s PPBS in school systems NY• 1968-80

– OD exercises related to goal and objective setting– Started doing evaluations asking what are your

objectives– Use of LFA for planning – some evaluation

• LFA based evaluations 1980s-90s – Project and Program Planning

• 1994 CIDA International Consultation with RBM-CIDA Training assignments

• RBM, Performance Management, Evaluation, Monitoring are code words for a wide range of assignments undertaken.


My Involvement in RBM Internationally

• CIDA-Facilitator, Coach, material designer, trainer• Asian Development Bank-Evaluator• World Bank – Facilitator, Coach, material designer, trainer (SRM, External

Affairs) Eval.2002-2005 – country program reviews• OECD – MfDR 1 writer and contributor• IUCN-RBM Evaluation System 2000-2006• Rockefeller Foundation – RBM Evaluation System 2007• UNHCR-UNHCR – Coach and Trainer for RBM system Communications 2005-

2007• Canada-South Africa Educational Management Program – Project designer and

director – RBM system 1995-2000• Almost all our evaluations now linked to RBM concepts and frames


OBSERVATION 2: CONSTANT PERSONAL CHALLENGES

Why are we still Questioning RBM?


Pictures of the year by NBC

RBM Efforts try to Align Organizational Pieces


A rational approach

An approach to change

A linear systems approach

A technical approach

But, there are other Organizational Logics

• For example – Lee Bolman and Terry Deal– Rational bureaucratic-RBM as a “technical

fix”– Symbolic – RBM as “the answer”– Human resource – RBM as a “learning” tool– Political – RBM as a ”power” tool


OBSERVATION 3: HOW LEADERS AND STAFF CREATE THEIR MENTAL MODELS OF HOW THEIR ORGANIZATION WORKS OR “SHOULD WORK” AFFECTS THE IMPLEMENTATION OF RBM

RBM Used at Multiple Levels

• Started at the project level – now applied to– Programs– Organizations and Institutions– Networks– Sectors– Countries


OBSERVATION 4: IT IS USED AS AN ALL PURPOSE FIX FOR ACCOUNTABILITY AND CREATING CHANGE “WE NEED TO

MANAGE FOR RESULTS” WHO CAN BE OPPOSED?

Brainstorming: Things I have Learned


Learning – Lots of Hypotheses that need Testing

• Long term approach• A tool not an answer• Stable env vs. unstable env • Sr management cues• Ownership--throughout• Results Culture (trust, praise, …)• Belief systems and trust• Show utility early-quick wins• Clarity of purpose• Leadership-champions• Support structures• Incentives• Stick too it – embracing error• Start simple tools (Bureau-kills)

• Experimentation• Know the basics • Flexible 80-20 rule• Language• Importance of results to the

implementers • Rolling up results takes maturity• The data is worse than you think or

they say • Transparency• Communicate, communicate,

communicate• Use organizational cycles• Entry points vary


Four Areas for Observations

1. Self reflection2. Doing it3. Leadership4. Staff and culture


Reflection – Some Tips

• It is about change – first and foremost• Keep a journal• Respect other people’s way of knowing• Get wide feedback • Listen for distress signals• Always aim high BUT 80-20 is critical• Don’t let excellence get in the way of good• Model what you teach – very hard• Know what you are doing – keep learning • Humility is called for – know what you do not know• Be a positive force• Provide secure learning environment


OBSERVATION 5: IT IS ABOUT “THEIR” PERFORMANCE IMPROVEMENT –NOT YOU OR RBM

Lessons about Leadership

• Vision matters• The degree of followership• Honest purpose for introducing RBM tools• Willingness to model and experiment • Demonstrate ownership • Embracing error• Belief system – mental models about organizations• Champion and encourages others to champion


OBSERVATION 6: LEADERSHIP MATTERS

Doing it

• Images, images, images• Time and timing are critical• It is more paper – but this should be good

paper• Keep your eye on outcomes• Outcomes are partnerships


OBSERVATION 7: NEVER ENDING – DO REFLECT – DO REFLECT

The Time to Change is Now!!


New Phase and Change

High Growth

Maturity

Decline

Strategic Change

Challenge 1: How Aligned Are We? In it Together!


Communication Office

UNHCRs Strategy

Strategic Objective Strategic Objective 2 Strategic Objective

3Strategic Objective

1

Outcome 2Unit Outcomes Outcome 3Outcome 1

Department 1 Department 2DepartmentResponsibilities

Business Lines Outputs Outputs Outputs Outputs

Mission Resources

PM&R Built on a Results Framework


Inputs

Activities

Outputs

Outcomes

Impacts

Zone 1: InternalHow we turn inputs and activities into outputs

Zone 2: Missing MiddleHow outputs contribute to Intermediate Outcomes Zone 3: Results monitoringMeasuring outcomes & Impacts

Results Framework… Zones of focus… Control vs influence…

Sphere of Control:Good data; attribution

Sphere of Influence:Contribution

attribution is much harder

Tips


JulAug

Sep

Oct

DecJan

Mar

Apr

May

Jun

Retreat

Key COA-Sr. Mgt PB&M

Key Divisional MeetingsOn PB&M

Feb

Nov

Key UNHCR Meetings

Challenge 2: Unit Plans


Planning and Unit ToolI. Overview of Unit

Short Description of Unit Role in COA Strategy: Rationale for the Unit (optional

Unit Strategic Objectives (Align with Strategy) Indicators of Success Yr 2005-

Challenges in meeting Objectives)

Unit Strategic Objectives 1

Outcomes Indicators FY 05-07 Outputs in FY Person(s) Responsible

Individual Work Plans

Outcome(If known) Outputs

Other staff Contributing to this output

Person responsible

Due Date


P, M & R System


Planning

Business UnitStrategy & Results

Annual Reports onStrategy & Business Units

Management Review Team

Communication Office

Quarterly Reports

Update Strategy and Business Plan

Annual Work

Program

Unit QuarterlyReports

UN

HC

R’s

Str

ateg

y &

Fra

mew

ork

Com

mun

icat

ion

Stra

tegy

Bus

ines

s Pl

an

Monitoring & Reporting

Unit

outputs outcomes)

Dra

ft –

Oct

ober

20

05

The Components

• Strategy – Business plan• Unit plans• Individual Work plans (linked to UNHCR’s

RCA)• System for creating data to monitor and

report on Office, Unit and individual plans –quarterly, annually and multi year

• Reports (Office, Unit and individuals)• UNHCR‘s Balance Scorecard


Lessons – Tips About Culture

• Collective stories, norms, values, symbols of People• People mental models and actions matter• Language matters – outputs – outcomes – impact• What gets rewarded- supporting others – effort –

results• Experimental – Learning needs to be a value• Time and norms• Moving from performance rules to performance

values


OBSERVATION 8: NEXT TO LEADERSHIP

It is About “Good” Public Organizations

• Determined leadership• Clear direction-concrete goals• Execution oriented management• Rigorous people practices• Performance oriented culture

Jerry Hauser and Alison Green “The Management Center


COME TO TALK WITH US AT:

THE HOSPITALITY SUITEAT THE WESTIN HOTEL

TONIGHT AT 9:00 PM!


rbm: international experience - evaluationcanada.ca

Documents