Introduction to the Pride Campaign ScorecardQuality Management & Learning TeamDaniel HaydenMarch 3, 2009

“Not everything measurable matters. Not everything that matters is measurable."

QM and Learning Process

QM & Learning Team Monitors and Supports Whole Process

Today’s Focus

What Questions Go Into the Scorecard CascadeCampaign Impact

Capacity Constituency Conservation

Leading (Operational)• Likelihood of achieving capacity

goals (WFR)• Campaign timeliness (Milestones)• Grades (WFR)• Partner satisfaction (360 survey)• CM capacity development (360

survey)

Lagging (Impact)• CM graduation (Pf) • CM active in conservation field

(survey TBD)• Partner increasing investment in

community engagement (survey TBD)

Leading (Operational)• Likelihood of achieving

constituency goals (WFR)• Stakeholders are discussing the

campaign (Monitoring Report)• Stakeholders are likely to

change behavior (Monitoring Report)

• Number of friends on RarePlanet (RarePlanet)

Lagging (Impact)• Change in Knowledge (Details

Page)• Change in Attitudes (Details

Page) • Change in IC (Details Page)

Leading (Operational)• Likelihood of achieving

conservation goals (WFR)• Quality of deliverables (WFR)• Fundraising

Lagging (Impact)• Achievement on SMART

objectives (Details Page)• Cost per Adoption (Details

Page)• #/% of target adopting new

tool (Details Page)

Community support to work towards threat reduction and environmental protection

The ability of the Campaign manager and local partner to run a successful social marketing Campaign

Combines the: 1. Reduction of threats /

changed behavior 2. 2. Improvement to the

local environment

How is the Score Calculated

Campaign Colors

Color Meaning Score

Green■

Campaign is on track to achieve goals in stated timeline

4.0 to 5.0

Yellow▲

campaign is at risk of not achieving some of its goals, or has fallen about a month behind schedule

2.5 to 3.99

Red

Campaign is unlikely to achieve its goals or is severely behind schedule

Below 2.5

Business Rule: •A Red score on any metric turns category Red•Red in any category (any of the 3Cs) turns whole campaign Red•Any data that is more than 3 months old, does not go into the overall score – this allows campaigns to recover from past mistakes (of course the best thing is the PPM to work with the CM and Partner to fix the issue).•Data that is missing does not go into average e.g. N/A does NOT equal 0•This override rule that turns campaigns Red can be switch on / off if we believe the measure is too harsh (we don’t want to pretend all our campaigns are great, but we don’t want it to appear that they are all failing either).

Business rules address the logic that governs the Scorecard. For this initial launch we have kept these rules relatively simple. Based on our experience, we may adjust them.

Learn & Improve- How Do We Use the Scorecard• Transparency - The results of the Scorecard are reported to the Board each

month• Management – We use the results of the Scorecard to better understand if

a campaign needs help or should be highlighted in success stories• Training – the Scorecard provides ideas on what sort of issues staff should

look for• Learning – We have and will more so periodically look at the top and bottom

campaigns to see what lessons can be learned• Cohort Management – The Scorecard helps determine who comes back

from to the second university phase and who graduates

Scorecard Design and Rollout

6

Design5/08 to 12/08

•How do we measure success•What are the key pieces of data•How little info can we ask for and still get meaning•How do we want to communicate

Design5/08 to 12/08

•How do we measure success•What are the key pieces of data•How little info can we ask for and still get meaning•How do we want to communicate

Build06/08 to 12/08

•Business rules•Data capture forms•Data flow•Reporting

Build06/08 to 12/08

•Business rules•Data capture forms•Data flow•Reporting

RP Team Test12/08 to 05/09

•Mechanically, does it work as designed•Do the results make sense across varied situations•Is the reporting clear•Is the user experience acceptable•Does everyone get value

RP Team Test12/08 to 05/09

•Mechanically, does it work as designed•Do the results make sense across varied situations•Is the reporting clear•Is the user experience acceptable•Does everyone get value

Learn & Adapt

Internal Beta2/09 to 06/09

•User experience•Does it provide insight into campaign success•Does answer other key ques.•Is the reporting clear and valuable

Internal Beta2/09 to 06/09

•User experience•Does it provide insight into campaign success•Does answer other key ques.•Is the reporting clear and valuable

Appendix

Gather Data Overview of RarePlanet – Campaign Management

Orientation

• Weekly Flash Report•Campaign Details Page

Planning

•Weekly Flash Report•Monitoring Report•Survey: Local partner, (Executive Director, Local supervisor), BINGO•Campaign Details Page

Implementation

•Weekly Flash Report•Monitoring Report•Survey: Local partner, (Executive Director, Local supervisor)•Campaign Details Page

Analysis

•Weekly Flash Report•Monitoring Report•Survey: Local partner, (Executive Director, Local supervisor)•Campaign Page

Post-campaign monitoring

(Monitoring partner)

Data Gathering Is Integrated Across the Life of Campaign

Everyone Contributes to the Scorecard (p. 1/3)Tool Objective/Purpose Doer Frequency System User

Flash Report Answering Questions:-How is the campaign going?-Are we on track to achieving conservation objectives?-Are there major concerns?-Provide a quick qualitative updateServes as a list of what we do at Rare

PPMValue: -Manage portfolio better-Manage communications with team-Ease of reporting

Weekly RP Pride Team (Directors, VPs)HQ

Monitoring Trip Reports

- On the ground validation of progress

- Should contain space for people to provide more details about the trip

- learning

Currently PPM, but should be anyone who visits an LAP and/or campaign site (e.g. Directors,, VP, HQ)Expand to make more relevant to users

Per trip to the site RP - Scorecard- Who will read the

qualitative info? We need to define a process

Everyone Contributes to the Scorecard (p. 2/3)Tool Objective/Purpose Doer Frequency System User

Surveys-CM-LAP-BINGO-Alumni

CM: Capture satisfaction with campaign, with RareLAP: Capture satisfaction with campaign, with CMBINGO: Capture satisfaction with campaign, plans for repeated partnerships with RareAlumni: Capture follow-up dataFor Development - Links to BSC (balanced scorecard) and Marketing – Prideforce

PPMs send surveys; Partners complete them-Automatic integration into scorecard

CM/LAP: 4x (once per phase)BINGOS; roughly annually (twice)Alumni: once per year

RP (private messages)

- Scorecard- Partner

ManagementWe need a process

to ensure that the team review results

Campaign Details & Learning Form

Collect Campaign Results*Should be by cohort• Capture key facts about the campaign, especially regarding the campaign site and TOC/barrier removal plans.

- PPMValue:Used by- Director

/VP- Recruit

ment- Alumni- QA

Initial setup by staff (QA);PPM monthly update (as new information

At Graduation

RP - Scorecard- Board - Research- Marketing- QA- Learning as a

team

Everyone Contributes to the Scorecard (p. 3/3)Tool Objective/Purpose Doer Frequency System User

Prideforece (CRM) -relationship management- documenting interactions/communication- some documents (e.g MoUs, contracts)- task management – relationships- escalation of partnerships issues- general sensitive issues

- VPs, Directors- PPMs- HQValue:CRM

As needed; monthly minimum

Prideforce - PPMs - Directors

Monthly Finance Monthly Project Financial Report

Replicon Monthly? MIP - PPMs- Directors- VP/HQ

Appendix

Data GatheringOverview of RarePlanet – Campaign Management

Background• RarePlanet is the central knowledge repository and project management

system for all of Rare’s campaigns• For the past 9 months there has been a team migrating, filtering and

consolidating Rare’s disperse information repository and placing it into RarePlanet

• Much of this data is public, however some information is private• Private information typically relates to detailed contractual information and

ongoing relationship information about partners and people• All information that could be of general interest to a conservation practitioner

is public, available for comment and downloadable• RarePlanet has extensive permissioning that allow different views of the site

according to roles such as Rare Staff, PPMs, CMs, LAP partner staff and general users

• The views taken for this presentation are the highest level permission, so all information is available

This Presentation Works Left to Right Across the Tops of the Campaign Page• Overview• Details• Forms & Reports

– Weekly Flash Report

• Milestones• Scorecard

Overview Tab

Every campaign has an overview page that tells the story via pictures, display of their strategy for success (TOC), and milestones.

Details Tab

Every campaign has a page that details its main objectives, target audience and goals.

These are update about every 1 month to reflect the most recent information on a campaign.

Forms & Reports Tab

Every campaign has a page in which the campaign’s PPM can manage:

•The key deliverables•The dates•360 Feedback surveys•Inputs into the Pride Campaign Scorecard

Additionally, the PPM can see a summary result of the Weekly Flash Report.

The Weekly Flash Report

The Weekly Flash Report is a form found under the tab “Forms & Reports. “ Rare’s PPMs complete these weekly. The weekly flash reports do not capture all relevant campaign data, but rather act as a leading indicator of success.

Milestones Tab

Under the Milestones tab is the CM’s one stop shop for submitting his campaign deliverables. This creates:

• One place to go for the CM to communicate his progress• One place for Rare staff to find campaign information• An automated, centralized and standardized way to see if the campaign is running on time•Color scheme and scoring help motivate/drive CM to complete things on time

The data posted here, once approved, is then published to the campaign Overview page and the Resource center as a contribution to the conservation community.

Scorecard Tab

Rare scores all campaigns across the 3Cs:•Capacity•Constituency •Conservation

This is the output of all the forms that are managed under “Forms & Reports”

The Scorecard updates real-time based on the most current information.

Currently the Scorecard is being fully tested and it is only available to Rare staff.

Still Being Teste

d