[1]

Handout 1 Sustainability Planning Workshop Pre/Post-test [To be handed out on the first day of the workshop.]

PRE-/POST-WORKSHOP QUESTIONNAIRE SUSTAINABILITY PLANNING WORKSHOP

Please circle the ONE best answer. 1) Which of the following topics are NOT included in the six components of the

Sustainability Framework? a. Community capacity b. Program outcomes c. Financial capacity d. Health service delivery e. Organizational capacity 2) What are the five steps of the sustainability program design process? a. System situational analysis, system assessment, capacity building, program planning,

and monitoring & evaluation. b. Development of results framework, sustainability scenario planning, system assessment,

program planning, and monitoring & evaluation. c. System situational analysis, sustainability scenario planning, system assessment,

program planning, and monitoring & evaluation. d. System situational analysis, sustainability scenario planning, program planning, process

evaluation, monitoring & reporting 3) Which of the following statements about the local system is FALSE? a. The local system defines the level at which evaluation can take place in a meaningful

way. b. A local system is defined at the beginning of the project and does not change. c. The local system refers to the local stakeholders and communities brought together to

map out their vision and goals. d. A local system has boundaries; some people and groups are included and some groups

might have to be excluded. 4) “Children will not die of preventable causes. They will find quality care in well-managed

health centers and will be cared for by well-informed families.” This is a good example of a:

a. Component goal b. Sustainability scenario c. Program outcome indicator d. Strategic objective e. Vision statement 5) Which of the following is NOT an area that the sustainability scenario should begin to

broadly identify? a. Key strategies for attaining the vision. b. Specific activities that address each sustainability component. c. Broad roles/responsibilities of stakeholders.

[2]

d. Capabilities that are needed in order to fulfill the roles. e. Supportive environmental attributes that will help achieve the vision. 6) Which of the following are important sub-components of the enabling environment? a. Governance and civil stability b. Human development c. Health services d. a & b only e. All of the above 7) Which of the following statements about sustainability monitoring and evaluation is

FALSE? a. Many of the indicators selected for the local system assessment will also be included in

the M&E plan. b. The M&E plan should measure elements that are outside of the scope of the project. c. The local system assessment plan should focus on evaluative indicators; monitoring

indicators should not be included. d. Local system assessment is typically conducted two to three times over the course of a

project. 8) Which of the following is NOT an important criterion for a good indicator? a. Multidimensional b. Valid c. Reliable d. Precise e. Timely 9) A(n) __________ is calculated from a(n) _________ in order to measure the impact of

each __________. a. Scale, score, component b. Score, scale, indicator c. Scale, score, indicator d. Score, index, component 10) Name two important adult learning principles when facilitating the Sustainability Planning

Workshop.

[3]

Handout 2 Definition of Sustainability

An emerging property within a local system which

allows interdependent actors to maintain and

improve the health status of the system’s

(vulnerable) population through negotiated and

coordinated social interactions, allowing the

expression of their respective and collective

capabilities.

[4]

Handout 3 (2 pages) Introduction to the Sustainability Framework Find the triangle labeled “The Sustainability Framework.” Place it in the center of your table. Find the triangle labeled “Health Outcomes”. Place it on top of the Sustainability Framework triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “health outcomes” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.

Component 1, Health Outcomes, does not quite measure the health status of the

population—as this would require measuring indicators of morbidity and mortality. This

is not feasible for most projects. To be the most feasible while still giving the most

useful information, this component tries to come as close as possible to health status

measures. So, the usual data used for Component 1 is a summary of the population

health outcomes (i.e., indicators of key household behaviors and service coverage) that

are known to be associated with a high impact on health. These are the very measures

that many projects use to track their progress.

Find the triangle labeled “Health Service Delivery.” Place it along the right side of the “Health Outcomes” triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “health service delivery” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.

Component 2, Health Service Delivery, addresses how well the local health providers—

both facility and community-based—deliver services and products to the beneficiary

population (i.e., quality of care, use of health management and information systems, and

proportion of anti-malarial stock-outs). This service delivery contributes directly or

indirectly to the health outcomes measured in Component 1.

Find the triangle labeled “Ministry of Health Organizational Capacity & Viability.” Place it on the left side of the Sustainability Framework triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “MOH organizational capacity and viability” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.

Component 3, MOH Organizational Capacity and Viability, refers to the capacity and

viability of the organization that will supply the institutional support for health service

delivery. Capacity refers to the range of functions that are necessary to the life of an

organization, its administration, and its ability to perform its mission. Viability includes

financial viability, but also encompasses the more general idea of securing access to the

inputs necessary to sustain the level of capacity and performance attained during the

project. Some examples of MOH organizational capacity and viability include training

for maternal and child health interventions, supervision for community workers, and

financial resources for maternal and child health.

[5]

Find the triangle labeled “Local NGO Capacity & Viability”. Place it along the right side of the “Ministry of Health Organizational Capacity & Viability” triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “local NGO capacity and viability” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.

Component 4, Local NGO Capacity and Viability, refers to the capacity and viability of

the organization that will supply the institutional support for the activities in the

community necessary to demand services and for household behaviors related to

relevant health outcomes. Capacity and viability are defined the same as in Component

3. Some examples of local NGO capacity and viability include governance and legal

structure, financial management, technical capacity for maternal and child health

programming, and networking with partners.

Find the triangle labeled “Community Capacity.” Place it on the right side of the Sustainability Framework triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “community capacity” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.

Component 5, Community Capacity, refers to a relatively wide range of actors—from

individuals and households to social networks and the community as a whole—through

which community members will assume direct or supportive roles in achieving and

sustaining positive health outcomes. Some examples of community capacity include

organization, participation, and financial management.

Find the triangle labeled “Enabling Environment.” Place it along the right side of the “Community Capacity” triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “enabling environment” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.

Component 6, Enabling Environment, refers to the essential social-ecological

environment variables that can either support or weaken gains in health. Some

examples of the enabling environment include health policy and government commitment

to expanding insurance coverage, human development, gender empowerment, and natural

environmental factors. Clearly, more can be achieved in a more favorable environment

than a less favorable one. If we are going to be accountable for progress toward

sustainability, we should be clear about the context in which the local system operates.

[6]

Handout 4 Defining the Local System

Local system: A network of people and institutions whose coordinated

actions will bring about sustainable positive health outcomes in a

population.

“Local system” refers to the local stakeholders and communities brought

together to map out their vision and goals for sustained health improvement in

the community. This local system also defines the level at which evaluation can

take place in a meaningful way. Examples of stakeholders in the Local System

include: villages, women’s associations, local authorities, rural development

associations, health district and health posts, local socially active NGOs, and

private sector partners.

An important question for project planners and stakeholders to ask is, “How

broad is the ‘local system’?” Consider answering this question in terms of three

aspects: (1) It is the level of bodies/stakeholders that can be feasibly brought

together; (2) it is the level at which assessment can be conducted (villages

surveyed, facilities assessed, institutions willing to examine their cooperation

and functioning); and (3) it is the level at which decisions can be made in

response to the sustainability assessment (for example, the national government

is usually not involved though its decisions might be very important for

components of the SF, in particular component 6).

A local system has boundaries: some people and groups are included; groups that

are too remote might have to be excluded; and some groups exclude themselves.

Finally, a local system can evolve: groups once excluded can be included as they

see the benefit of the project’s efforts. Ultimately, a stronger and more

cohesive local system can help sustain better and better health outcomes.

Source: Yourkavitch, J., et al. Lessons learned from applying the CSSA to seven Mother and Child Health Projects.

[7]

Handout 5 Stakeholder Involvement Analysis Matrix

P

OW

ER

/IN

FL

UE

NC

E O

VE

R V

ISIO

N/

MIS

SIO

N

L

OW

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

--H

IGH

INFORM & ANSWER CONCERNS

CORE OF THE SYSTEM CHANGE AGENTS

MINIMAL INVOLVEMENT

KEEP INFORMED & FIND OPPORTUNITIES FOR COOPERATION WITH LOW TRANSACTION

COST

INTEREST IN VISION/MISSION

LOW------------------------------------------------------------------------------------------HIGH

[8]

Handout 6 Examples of Vision Statements

Children will not die of preventable causes. They will find

quality care in well-managed health centers and will be

cared for by well-informed families.

All people living with HIV and TB will be able to live healthy

and productive lives. They will access quality health

services and information in a timely manner and within a

supportive environment.

A malaria-free community where people don’t die because of

malaria and incidences of malaria are significantly reduced.

Every community will take a leading role in malaria

prevention, have equal access to effective interventions,

and receive coordinated support from stakeholders led by

the Government.

[9]

Handout 7 Example of a Sustainability Scenario

Vision:

All people living with HIV and TB will be able to live healthy

and productive lives. They will access quality health

services and information in a timely manner and within a

supportive environment.

Sustainability Scenario:

To attain our vision of improved health and well-being of

people living with HIV and TB, we will provide high-quality

services by building the capacity of service providers in

prevention, care and treatment services; strengthen village

health teams’ capacity to provide counseling and referral

services; strengthen supply management systems; improve

community members’ awareness and knowledge of HIV and

TB and the services available to them by mobilizing

community groups, local village committees, and CBOs/FBOs

to provide outreach and education. We will advocate for

supportive policy to allow for more resources to target

services for vulnerable populations that are at a higher risk

for HIV.

[10]

Handout 8 Plan’s Results Framework

Results Framework: Kilifi KID-CARE Project

Intermediate Result 1.1

Improved nutrition

in children

Result 1.2

Increased malaria prevention

and improved treatment

Intermediate Result 1.3

Increased immunization coverage

Intermediate Result 1.4

Improved case management

of pneumonia

Intermediate Result 1.5

Increased awareness of HIV/AIDS

and preventive behaviors

Result 1

Improved household behaviors

and management of childhood illnesses

through IMCI

Intermediate Result 2.1

Dispensaries have health workers

trained and equipped to manage

childhood illness

Intermediate Result 2.2

Villages have CHWs that are adequately

trained and equipped to do prevention

activities and manage basic illnesses

Intermediate Result 2.3

Increased coverage of outreach services

to hard-to-reach areas

Intermediate Result 2.4

Equitable Health Services

to all OVCs

Result 2

Increased access to quality

maternal and child health

services to OVC

Intermediate Result 3.1

Improved capacity of the MOH to offer

consistent and quality child-centered

services

Intermediate Result 3.2

Improved capacity of DHCs and VHCs to

finance and carry out health activities

Intermediate Result 3.3

Improved capacity of local CBOs/NGOs

to carry out quality child survival

activities

Result 3

Improved capacity of local partners and

established systems and structures that allow

for sustained CS activities

Sustained reductions in morbidity and mortality

of children and WRA in Kilifi District, Kenya

[11]

Handout 9 Plan’s Sustainability Framework Dashboard

[12]

Handout 10 (2 pages) Plan’s Local System Assessment Plan (Modified from Plan Kenya’s KIDCARE Project)

Component Indicators Measurement Method/Frequency

Person Responsible

1. Health Outcomes

Percentage of children 0-23 months that are underweight

Knowledge, Practices, and Coverage (KPC) survey/baseline, mid-term and final

Project Monitoring & Evaluation (M&E) Specialist

Child spacing

Skilled attendant at birth

Tetanus Toxoid coverage

Exclusive Breastfeeding

2. Health Services

Health worker (HW) correctly assesses for danger signs

Health Facilities Assessment (HFA)/baseline, mid-term and final

Project Staff

HW gives appropriate treatment for fever

Caretakers know 2 aspects of home care

Caretakers knows 2 signs of getting worse

HW is trained

3. Ministry of Health Capacity

4. Ministry of Health Viability1

District Health Committees (DHCs) have ability to analyze and use information generated by project

Custom capacity assessment tool/baseline, mid-term, and final

Project Capacity Development Specialist

DHCs know how to disseminate knowledge about nine key household/community child survival behaviors

DHCs know essential package of activities and supplies of the health facility

DHCs have a clear vision and mission and understanding of those

DHCs have the ability to motivate, mobilize and organize community

5. Community Capacity

Community has the ability to collect baseline information on any project before embarking on any work

Custom community capacity assessment tool/baseline, mid-term, and final

Project staff

Care group members knows how to disseminate knowledge about nine key household/community child survival behaviors

Village Health Committee (VHC) has the ability to motivate, mobilize and organize the community for health action

1 It should be noted that the previous version of the Sustainability Framework separated out Organizational

Capacity and Viability. We now combine these two aspects, but separate out the type of organization to which it is applied.

[13]

VHC has a constitution that governs its operations

Community members were involved in the formation/creation of the VHC

6. Enabling Environment

Female literacy rate National statistics/baseline, mid-term, and final

Central government

Number of demonstrations, riots, clashes or strikes in the district

Local statistics/baseline, mid-term, and final

local police or district security committee

% of women with cash income (rate) KPC survey/ baseline, mid-term, and final

Project M&E Specialist

Water and sanitation - Latrine coverage

KPC survey/ baseline, mid-term, and final

Project M&E Specialist

State of secondary roads Local statistics/ baseline, mid-term, and final

Local municipal authority or department of public works or transport

Project stakeholders meet after each measurement point to discuss findings and determine course correction, if needed.

[14]

Handout 11 (6 pages) Data Availability for the Sustainability Framework

Component 1: Program Outcomes

Data already available Where other useful data

may be available

New data collection

instruments necessary

[15]

Component 2: Health Service Delivery

Data already available Where other useful data

may be available

New data collection

instruments necessary

[16]

Component 3: Organizational Capacity for Service Delivery

Data already available Where other useful data

may be available

New data collection

instruments necessary

[17]

Component 4: Organizational Capacity for Community Mobilization

Data already available Where other useful data

may be available

New data collection

instruments necessary

[18]

Component 5: Community Capacity

Data already available Where other useful data

may be available

New data collection

instruments necessary

[19]

Component 6: Enabling Environment

Data already available Where other useful data

may be available

New data collection

instruments necessary

[20]

Handout 12 Sample Vision and Component Goals

CRWRC’s Malaria Communities Program in Malawi

Our vision is a malaria-free community where people don’t die because of malaria and

incidences of malaria are significantly reduced by 2020. Every community will take a

leading role in malaria prevention; have equal access to effective interventions (e.g., LA,

SP, ITNs and environmental control); and receive coordinated support from

stakeholders led by the Government.

Component 1 Goal: Significantly reduce malaria related morbidity and mortality

amongst communities by 2020.

Component 2 Goal: Ensure equal access of preventive measures and early treatment of

malaria through uninterrupted provision of necessary services and supplies.

Component 3 Goal: District level malaria staff and local health facilities will be capable

to provide adequate supplies, financial resources and well trained and skilled personnel

to assist in malaria prevention and treatment.

Component 4 Goal: Ensure community empowerment and stakeholder networking

through management of God-fearing, capable and motivated staff who work together as

a team.

Component 5: Ensure that the community will take a leading role to mobilize, manage

and utilize resources to prevent and treat malaria.

Component 6 Goal: Have a conducive environment where civil society is involved, there

is good governance, and proper policies in place to enhance provision of effective

malaria interventions.

[21]

Handout 13 Local System Assessment Plan

Component Goal Indicators Data Sources/Frequency of

Measurement

[22]

Handout 14 (3 pages) HealthPartners Collaborations Project Monitoring & Evaluation Plan (Revised for Workshop)

Performance Indicator Title Indicator definition and unit of measurement Method of Data

collection Disaggre-

gation

Frequency of Data

Collection

Strategic Objective 30,000 WRA (85,000 total beneficiaries) have access to quality health care through financially sustainable community owned health co-ops.

1 # of people2 covered by USG-supported health financing arrangements

Component 1: Program Outcomes

Definition: Number of people reached with health cooperative financing program affiliated to CDP during the reporting period. Unit of measure: Number

Scheme performance

tracking tools and METRICS

3 (Q22)

Gender: Male

Quarterly

2 # of cooperative members who are female

Component 1: Program Outcomes

Definition: Total number of women registered as cooperative members during the reporting period. Unit of measure: Number

Scheme performance

tracking tools and

METRICS (Q23)

Gender: Female

Quarterly

Objective 1: Annual stakeholder action plans for 6 co-ops detail public/private partnerships for improved health

3 # of stakeholder workshop reports detailing public/private partnership action for improved health

Component 4: Organizational Capacity (NGO)

Definition: Number of Health coops with stakeholder workshop reports detailing public/private partnership action for improved health. Unit of Measure: number.

Minutes from annual

stakeholders’ workshops

None Semi-

annual

IR 1.1: 42 Cooperative member groups elect providers based on measures of quality and establish MOUs

1.1.1 # of Coop member groups affiliated to CDP

Component 3: Organizational Capacity for Service Delivery

Definition: Total number of coop member groups formed with support from CDP Unit of measure: Number

Scheme performance tracking tools

None Quarterly

1.1.2 # of co-op providers who signed MOUs with member groups

Component 3: Organizational Capacity for Service Delivery

Definition: The total number of providers who signed MOUs with member groups. Unit of Measure: Number

Scheme performance tracking tools

None Annual

2 Number of people of people to be reached excluding Women of Reproductive Age are 55,000, but also women of Non Reproductive Age are included

3 Measurements for Tracking Indicators of Cooperative Success (METRICS) Questionnaire for Co-ops, OCDC and USAID 2009

[23]

Performance Indicator Title Indicator definition and unit of measurement Method of Data

collection Disaggre-

gation

Frequency of Data

Collection

1.1.3 %of providers with improved ratings on their member quality assessment scores

Component 2: Health Services

Definition: Members conduct annual provider quality assessments of their providers. Numerator: Number of positive ratings Denominator: Number of questions. Unit of Measure: %

Provider quality assessments

shared at AGMs

By Provider

Annual

IR 1.2 6 health co-ops sign annual MOUs with five partners each

1.2.1 # of VSLA cooperative members

Component 5: Community Capacity

Definition: Total number of VSLA members registered with Health cooperatives affiliated to CDP during the reporting period. Unit of Measure: Number

Co-op scheme performance

tracking

Gender: Male & Female

Quarterly

1.2.2 # of VHT cooperative members

Component 5: Community Capacity

Definition: Number of VHTs registered with CDP supported health cooperatives during the reporting period. Unit of Measure: Number

Co-op scheme performance

tracking

Gender: Male & Female

Quarterly

1.2.3 # of women’s organizations

Component 5: Community Capacity

Definition: Number of women’s organizations registered with CDP supported health cooperatives during the reporting period. Unit of Measure: Number

Co-op scheme performance

tracking None Quarterly

1.2.4 # of drama group cooperative members

Component 3: Organizational Capacity for Service Delivery

Definition: Number of drama group members registered with CDP supported health cooperatives during the reporting period. Unit of Measure: Number

Co-op scheme performance

tracking

Gender: Male & Female

Quarterly

Objective 2: Six health cooperative boards maintain their cooperative cost balance with annual provider surpluses and reserve fund

2.1 Total annual average provider surplus by co-op

Component 4: Organizational Capacity (NGO)

Definition: Total annual provider surplus Unit of measure: Amount

Co-op scheme performance

tracking By co-op Annual

2.2 Total annual co-op reserve fund balance

Component 4: Organizational Capacity (NGO)

Definition: 10% of the total annual provider’s surplus Unit of measure: Amount

Reserve fund bank account

statement By co-op Annual

[24]

Performance Indicator Title Indicator definition and unit of measurement Method of Data

collection Disaggre-

gation

Frequency of Data

Collection

IR 2.1 18 private health care providers employ data entrants to track monthly co-op performance

2.1.1 # of providers that turn in co-op performance tracking tools.

Component 4: Organizational Capacity (NGO)

Definition: Number of Health facilities affiliated to CDP that turn in tracking tools which they use to inform decision makers during the reporting period. Unit of Measure: Number

Co-op scheme performance

tracking By co-op Quarterly

IR 2.2 Six health cooperatives member elected boards of directors meet routinely to review performance

2.2.1 Average percentage of self-assessment score by the board members

Component 4: Organizational Capacity (NGO)

Definition: Understanding of roles and responsibilities as measured by average percentage score on self-assessment score sheet. Unit of measure: %

Review Board Assessment

forms By co-op Quarterly

2.2.2 # of performance review meetings held by Co-op boards

Component 4: Organizational Capacity (NGO)

Definition: Total number of performance review meetings held by CDP supported co-op boards in the reporting period Unit of measure: Number

Review of co-op board meeting

minutes

By Health co-op board

Semi-annual

2.2.3 # of people attending co-op performance review meetings

Component 4: Organizational Capacity (NGO)

Definition: Total number of people that attended coops performance review meetings Unit of Measure: Number

Review activity reports

Gender: Male & Female

Semi-annual

Objective 3: One CDP partner collaboration plan developed and lessons learned in cooperative health financing shared with MOH and partners

3.1 Number of CLARITY related collaborative activities(workshops, conferences, seminars…etc)

Component 6: Enabling Environment

Definition: Number of CLARITY related collaborative activities(workshops, conferences, seminars…etc) Unit of measure: #

OCDC None Annual

IR 3.1: HealthPartners annual reports document how lessons learned were shared with international and national health partners

3.1.1 Number of METRICS related collaborative activities (workshops, conferences, seminars…etc)

Component 6: Enabling Environment

Definition: Number of METRICS related collaborative activities (workshops, conferences, seminars…etc) Unit of measure: #

OCDC None Annual

3.1.2 Number of IMPACT related collaborations (workshops, conferences, seminars…etc)

Component 6: Enabling Environment

Definition: Number of IMPACT related collaborations (workshops, conferences, seminars…etc) Unit of measure: #

OCDC None Annual

[25]

Handout 15 Health Facility Assessment Indicators

Area # Domain Indicator

Access

- Geographic Access % population with year-round access to MNC services

1 Service availability % HF in which MNC services are available (Child: sick child, immunizations, GMP; MNC: ANC services)

Inputs

2 Staffing % staff in HF who provide clinical services and are working on the day of the survey

3 Infrastructure % essential infrastructure in HF to support MNC services available on the day of the survey

4 Supplies % essential supplies in HF to support MNC services available on the day of the survey

5 Drugs

% first line medications for MNC services available in HF / CHW on the day of survey (HF: ORS, oral antibiotic for dysentery, oral antibiotic for pneumonia, first line anti-malarial, vitamin A / CHW: context-specific)

Processes

6 Information System % HF/CHW that maintain up-to-date and complete records of sick U5 children / ANC services AND show evidence of data use

7 Training % HF/CHW where interviewed HW reports receiving in-service or pre-service education in MNC in last 12 months

8 Supervision

% HF/CHW that received external supervision at least once in the last 3 months (includes at least one: check records or reports, observe work, give feedback)

Performance

9 Utilization # sick child visits per year per U5 child in HF catchment area

10 HW Performance:

Assessment

% HF in which all essential assessment tasks were made by HW for sick child (pass = 80% observed cases)

11 HW Performance:

Treatment

% HF/CHW in which treatment was appropriate to diagnosis for child with fever, ARI, and/or diarrhea (pass = 80% observed cases for HF / 80% most recent cases in register for CHW)

12 HW Performance:

Counseling

% HF in which caretaker correctly describes how to administer all prescribed drugs for malaria, ARI, and/or diarrhea (pass = 80% exit interviews)

[26]

Handout 16 (2 pages) Somota and Jamuna Group Stories

SSoommoottaa WWoommeenn’’ss GGrroouupp SSttoorryy

The Somoto women’s group is a group of twenty women from Lalbagh slum in Dhaka. This group

was formed with the help of Sathi staff in 1999. This group of women began by saving about

ten cents a week. They would meet once a week to collect savings, to talk about activities they

could do, and to make plans. Nioti, the Sathi staff, worked with them to develop leadership and

committees within the group.

In the first year, all of the 20 women were involved in literacy classes. 16 of them were able to

pass the advanced literacy class, putting them at about a 5th grade level of reading and writing.

Right away, they began to take over the tasks of writing in the passbooks, the meeting minutes,

loans etc. They had elections each year to change the leadership. The group realized that they

needed more than one person to do the writing, the meeting leading, the monitoring of income

generation, etc. So, besides the main person, they had group members learn the tasks as

trainees and then each year a new trainee would take over the activity. The group listed each

activity they did. Their goal was to have each of the 5 women trained in each activity. The

activities included things like maintaining passbooks, weighing children, doing banking, leading

meetings, teaching literacy to new members, auditing accounts, etc. Occasionally one of the

women would move away to another area and have to leave the group. However, her

responsibility is taken over by someone else. In 2005, the leadership had changed through

elections.

QUESTIONS:

According to this story, how would you define capacity?

Identify some of the activities/events that promoted capacity development?

What are some of the things groups or communities in the Collaborations Project do to

build capacity for sustainable groups or communities?

[27]

JJaammuunnaa WWoommeenn’’ss GGrroouupp SSttoorryy

Pari is a rural development project located in Jamalpur, Bangladesh. It works in integrated

community development programs in five subdivisions of Jamalpur. The programs include adult

non-formal education (literacy), health promotion, agriculture, savings based income generation,

leadership development and small business development. The foundation of the program is the

development of primary groups in the community, separate for men and women. These groups

consist of 20-25 individuals who are involved in the various activities. Field staff are hired by

Pari to live in the communities and help to form and develop the primary groups.

Jamuna is a Bengali woman who was hired by Pari in 1994 to move with her family to Nandina,

about 20 km from Jamalpur town. She began by relationship building, Participatory Rural

Appraisal and then began with having the women save weekly.

Jamuna was a very enthusiastic staff and 10 women’s groups were formed in the community.

The groups met weekly for their savings meetings. Jamuna never missed a meeting – rain or

shine she was there. She helped the groups to form bank accounts and each week she brought

the savings to the bank. As the women took literacy, they learned to be able to keep the

records. Each month Jamuna took the record books home and checked them all and made the

corrections and sent the books back to the women’s groups in the villages. Once a year she

audited the accounts book of each group. She corrected mistakes and then met with them to

review the status of their accounts.

The group members learned literacy. Jamuna taught the classes which were for six months

after the group was formed. The literacy classes met every day for the six months. Some of

the women learned to write the minutes. Jamuna would check them each week and made

corrections.

In health, Jamuna taught health lessons each week. She would take the women to local clinics

when they or their children were sick. The women would come to her house on weekends as well

if they needed advice on health issues. Jamuna also started a growth monitoring program and

[28]

weighed the children in each village every month. Jamuna had the mothers keep the weight

cards and she explained the weight to them each month.

When there were land and other disputes in the community, Jamuna was often called on to help

mediate the disputes. When there were arguments about which income generation project to

start, or problems with money, Jamuna would solve those problems.

After Jamuna had been working in Nandina for six years with these ten groups, her supervisor

came and shocked her with the news that she was being transferred and needed to leave

Nandina in two weeks. Jamuna and the women were devastated. The supervisor explained that

since Jamuna had worked with these same groups for six years, she could now leave and these

groups would continue.

It was a sad time. Jamuna only had a few days, so she went to each of the groups and explained

to them that they had learned a lot and she trusted that now they could continue all of their

work on their own and continue with their development. The group members were doubtful.

Jamuna had done so many things and they were frightened.

Jamuna left and moved to another area to work. Three years later she went back to Nandina to

visit with the women’s groups.

QUESTIONS:

What did Jamuna find when she went back to Nandina 3 years later? Do you think

the groups were still functioning? Do you think they were strong?

What do you see were some of the mistakes Jamuna probably made when working with

the groups?

What are 5-6 things Jamuna could have done during her working time to help to

ensure that capacity is built for more sustainable groups?

What are the positive steps you take in the Collaborations Project to help ensure that

capacity is developed?

[29]

Handout 17 (4 pages) Organizational Capacity Indicators

Area Domain Indicator

Governance

Legal Recognition The organization is a legally constituted entity, recognized by the host country government

Governing Committee or Board

Organization has a governance body that meets and makes decisions that guide the organization's development

Constitution / Bylaws Organization has a written constitution and/or bylaws accepted and approved by the governance body

Accountability/Integrity Organization has policies/procedures in place to minimize conflicts of interest among leaders and staff through disclosure of conflicts.

Mission and values Organization has a mission and set of values which are clearly understood, agreed and approved by all the members of the organization and these are used

Transparency of Decision-making

There is a systematic process so that decisions are made by senior leaders and the governance body in a way that all in staff are aware of them and understand

Organizational Structure

There is a clear organizational structure in place, with clearly defined roles and effective coordination and communication among departments.

Participation in Decision-Making

(Board/Senior Leadership with all

staff)

Everyone in organization feel that they have been sufficiently consulted and their concerns addressed for important decisions

Human Resources

HR Management Capacity

The organization has an adequately resourced HR function, with professional qualified HRM staff, and clear plans & budgets for HR activities the HR function plays a valued &strategic role.

Staff Roles & Responsibilities

Staff roles and responsibilities are clearly written and well understood

HR Policies & Procedures

The organization has detailed formal HR policies & procedures

HR Personnel Files The organization has up-to-date, accurate & secure employee data.

Staff Welfare &Morale The organization has mechanisms to promote good employee relations and ensure high staff morale.

Staff satisfaction Staff feel satisfied and well treated by the organization

Remuneration System A formal remuneration policy, job classification &salary scale is in place.

Recruitment There are effective systems to support high quality staff recruitment.

Performance Management

The organization has formal systems for performance management.

[30]

Area Domain Indicator

Project Management

Beneficiary Targeting Each project formally defines its beneficiaries. There is a systematic process to ensure that all beneficiaries receive the outputs of the projects.

Project Planning Organization plans for and maps out individual projects that fall under the overall organization work plan

Project Budgeting Organization practices project- level budgeting, monitoring and evaluation.

Monitoring & Evaluation

Resources and a foundation for monitoring and

evaluation systems

There are secure resources (staff, tools, etc.) to carry out M&E activities and a monitoring and evaluation strategy with indicators and targets has been established.

Monitoring data collection and storage

Data are regularly and systematically collected and maintained.

M&E data informs decisions

Monitoring and evaluation data are systematically used to inform program and management decisions.

Data Quality Assurance

Data quality is measured systematically and on a routine basis and quality issues identified are rectified.

Financial Management

Financial accounts/bookkeeping

Organization keeps accounts of money that can be presented on demand

Internal Controls Organization has designed appropriate internal controls and controls are operating effectively

Payroll System Organization is able to correctly calculate payroll as well as remit all appropriate amounts to employees and to the taxing authority

Accounting policies and procedures

Organization has documented all applicable policies and procedures

Bank account Organization has a secure bank account to hold its funds

Record Keeping Organization maintains organized and standardized supporting documentation for every expenditure

Organizational Budgeting and

Financial Planning

Organization prepares, monitors, and updates its corporate budget on a regular basis, with input from all departments

Development and management of donor

agreement budgets

Organization prepares, reviews, and updates donor agreement budgets consistently and accurately.

Financial reporting/Donor

Reporting

Organization provides high quality and timely financial reports as required

[31]

Area Domain Indicator

Technical Capacity

Staff orientation Organization has a systemic process for orienting new staff and preparing for their new responsibilities.

Staff development and training

Organization has a systemic process for orienting new staff and preparing for their new responsibilities.

Access to technical resources

All technical/program staff have and use access to technical resources necessary for their work.

Office Operations

Procurement System Organization has a procurement system that maximizes competition, obtains best value, and observe compliance requirements.

Security and Safety Management

Organization has a security system that effectively identifies and manages all physical and staff security.

Physical infrastructure Buildings and office space: Organization has sufficient office space, meeting space, and equipment for handling its business.

Technological infrastructure

Telephone/fax: Organization has sufficient telephone and fax facilities which allow for efficient and effective communication.

Motor Vehicles Management

Organization has a system that manages, protects and controls motor vehicle usage.

Travel Management Organization has a travel management system that efficiently manages both domestic and international travel.

Records and Information

Management

Organization has a management system that manages records filing, retention and archiving, observing legal and compliance requirements.

Information Technology

Technological infrastructure

Computers, applications, network and internet facilities: Organization has state of the art fully networked and integrated computing resources. Organization maintains servers for key applications either in-house or via a hosted service

Software Systems

Systems are in place which allow the organization to track, report, and transform Contracts & Grants and Program data. Organization is using a formal accounting system.

Communications

Branding/Messaging Organization has an official name, a mission statement, a logo, and clearly defined messages to communicate its mission to the public.

Media/Public Relations

Organization has a media strategy, including a process to identify and communicate with media (including print, broadcast/radio, and online), respond to media requests, and produce press releases/ statements.

Brochures/Collateral Materials

Organization has a well-designed set of collateral materials for marketing purposes, and the capacity to produce reports and other publications when needed.

[32]

Area Domain Indicator

Program Management

Strategic planning Organization has a strategic plan (an all-encompassing document that guides its programming and aspirations)

Annual Organizational Work plan

Development

Organization plans the development of its activities, involving all relevant staff and stakeholders

Participation in Decision-Making

(within Units/Departments)

Everyone in unit/ department feels that they have been sufficiently consulted and their concerns addressed for important unit and department decisions

Quality Improvement System

There is a process to use information-driven approaches to improve organizational learning and performance (at all levels--technical, programmatic, etc).

Annual workplan completion

The organization regularly checks progress against the workplan and revises plan as necessary.

Resource Mobilization

Identification and tracking of funding

opportunities

The organization has a systematic manner of identifying and tracking new funding opportunities in order to make strategic decisions.

Organization has the capacity to develop funding proposals

There is satisfactory capacity to write, budget, partner and manage the proposal development process.

Success rate in winning bids

Organization has been successful in raising donor funds.

Resource diversification

Organization relies on a diversified resource base.

Networking

Networking to coordinate with other

non-governmental implementers

Organization has relations with other implementers to include CBO's and community based actors in order to coordinate service delivery and avoid duplication of services.

Collaborating with relevant government

agencies

Organization has relations with government entities, for coordinated implementation and/or advocacy for policy change

Partnerships for technical assistance

Organization has partnerships for provision of technical assistance from agencies that include: UN agencies, large NGOs, local universities, and others)

Relations with potential donors for

funding

Organization maintains relations with a diverse set of potential donors and keeps them informed of its work, so that it can efficiently take advantage of funding opportunities as they arise

[33]

Handout 18 (2 pages) Community Capacity Indicators Indicators on Community Capacity to Respond to Burden of Disease (Note that many of these indicators can be obtained through standard KPC surveys. The indicators proposed here need to be further specified according to priority disease areas affecting the community.)

Area Indicator

Perception of the Disease

Perceived threat of the disease on the community

Perceived role of the community confronting the disease (Extent to which the disease is accepted as a “fact of life” versus something the community is empowered to combat)

Knowledge about the Disease

Awareness of existence/presence of disease in the community

Knowledge of how disease is transmitted

Knowledge of how disease is prevented

Knowledge of available treatment for disease

Response to disease in the

community

An action plan to combat disease and ensure health of its members has been developed by the community

The community reaches out to support those who are vulnerable to disease

Access to primary health facility community members

Access to outreach services (in the community) by community members

Use of recommended preventive measures by community members

Use of recommended treatment by community members

Awareness of Gender Issues

Community members are aware of specific gender roles that may have an effect on burden of disease

Community members take action in line with gender roles.

Measuring Change in the Community

The community has articulated the changes it would like to see in the community with respect to its health

The community has put in place a method of measuring progress towards these changes

The community is able to measure changes and demonstrate improvements in the health of its members

[34]

Area Indicator

Learning & Sharing

The community is receptive to learning good health practices from outside sources

The community shares and applies these learnings

The community seeks out people with relevant experience to help them

The community shares its learnings and successes with other communities and also invites other to share their experiences

Mobilizing resources

The community is aware of the resources necessary to achieve their health improvement goals

The community acknowledges their role in mobilizing these resources

The community mobilizes internally to gather a portion of the needed resources

The community harnesses existing capacity to identify potential external resources and mobilize them.

[35]

Handout 19 List of Gender Questions

Integrating gender into a sustainability plan requires a gender analysis of key

stakeholders, including the implementing organization. It is important that

HealthPartners themselves works toward gender equality within the Uganda office, in

order to effectively integrate gender considerations into programming and capacity

building of partners.

The following are key issues to be identified in a gender analysis:

Practical needs of women, men, girls, and boys. (I.e. access to health commodities.)

Strategic needs of women, men, girls, and boys. (I.e. collective organization.)

Decision-makers

Differing roles and responsibilities of women and men, both within the household

and outside of the household/in the wider community

Beliefs, customs, politics, and tradition that could influence gender equality or

inequality

The following are some questions to consider when identifying the points above, and can

be integrated into existing capacity assessment tools:

Are women accepted as decision-makers?

Are women’s organizations present in the community? What activities are they

involved in; what issues do they address?

Who does what, when, where, and for how long in the community?

o Are there differing roles and responsibilities among women and men

within the household and outside? If so, what are they?

o Do beliefs, customs, politics or tradition that women and men hold

influence roles and responsibilities?

Who appears to be accessing services?

Who has access to productive resources like land, markets, equipment, capital,

credit, education, and training?

o How do beliefs/customs/politics/tradition influence access and control

of resources?

Who appears to have decision-making power (in households and in the

community)?

What practical needs of women, girls, men and boys are being met? Which are

not?

What are the opportunities for women, girls, men and boys? This could focus on opportunities for accessing health services, or having good health outcomes.

What are constraints for women, girls, men and boys? This could focus on constraints to accessing health services, etc.

[36]

Handout 20 Modified Gender Analysis Matrix

Below are some categories to consider when thinking through possible issues that influence behaviors and access in your program area. These categories are to assist you with putting on a “gender lens,” but are just a guide; there may not be a significant issue in every category, and there may be multiple issues under a single category.

What are the key

gender issues and

facilitators in each

domain below, in the

program area?

What other

potential

information is

missing, but

needed?

What are gender-

based constraints

and/or opportunities

for programming?

Potential strategies

to address issues or

capitalize on

facilitators

Practices, roles, and

participation:

Women cannot leave livelihood or other children for care seeking

Knowledge, beliefs,

perceptions:

Poor treatment by health facility staff

Access:

Distance to health facility

Assets (for transportation, fees at facility)

Low literacy

Legal rights and status:

Free ANC services in all public facilities

National Legislation Against Domestic Violence

Power and decision-

making:

Unequal representation (gender, ethnicity, etc) in community structures

[37]

Handout 21 (2 pages) Sub-Components of the Enabling Environment

Component 6—Enabling environment

There are essential social-ecological environment variables that can either support or weaken

gains in health. Clearly, more can be achieved in a more favorable environment than a less

favorable one. If we are going to be accountable for progress toward sustainability, we should

be clear about the context in which the local system operates. The stakeholders who need to be

part of the functioning of the local system, the strategies for achieving effective service

delivery and positive health outcomes, the social arrangements, the distribution of roles among

partners, and consequently what capacity and viability needs to be built within partner

institutions are only meaningful within a given environment.

Some environmental factors may be within the ability of a project or local system stakeholders

to influence, whereas others will not be. For instance, a project might identify partners to

advocate for policy change, but on the other hand, vulnerability to drought, food insecurity, and

other disasters will be difficult to mitigate. As a more specific example, if a project promotes

community case management of malaria, a national policy that prohibits use of anti-malarials by

anyone other than a licensed doctor is an “environmental factor” that will constrain the

sustainability of any gain achieved locally. It would be still worse if there was not even a policy

recognizing the legitimacy of community health agents in general.

Given the vastness of the social-ecological environment, you can feel overwhelmed by the task

of addressing it at all. But the task does not have to be daunting. We propose to guide you

through an “environmental scan” that includes six subcomponents. Before presenting them, you

must remember that the detailed content of the subcomponents for your project will ultimately

depend on the local context. But rather than start the discussion of the environment with

partners on a blank page, we suggest that you start with the description of the content of the

six subcomponents presented below.

1. Health Policy and Government Commitment to Health

The specific policy issues to be addressed will depend on the technical focus of the project.

But clearly, the level of commitment and resources devoted to health will be a major factor

in what is possible to implement and sustain.

2. Governance and Civil Stability

There are various measures to ascertain whether government institutions function and are

trusted. This is critical for support of government-sponsored health services and for the

climate in which civil society operates. Specifically, in terms of civil stability, in areas where

there is disruption because of war or insurrection, there is likely to be disruption of

services, strains in social networks, and even physical displacement of people. Disruption of

this sort makes gains in health tenuous at best. When the situation is at its worst, perhaps

immediately post-conflict, project managers should be cautious in terms what they promise

with regards to sustainability.

[38]

3. Human Development

Large swings in the economic and development landscape can make for shifting priorities

among organizations and individuals that may threaten health gains. For instance, in a

situation of drought or even famine, subsistence will predominate over health in terms of

national, local, and household priorities. The United Nations Development Programme (UNDP)

computes a summary measure of human development—the Human Development Index

(HDI)—for all countries on a periodic basis. UNDP country offices sometimes compute this

score for sub national regions as well. Progress on the HDI supports greater expectations

for what can be sustained.

4. Women’s Empowerment

The role of women is critical in population health gains, as women are the main caregivers to

themselves and their family members. Values related to women, in terms of their decision

making authority and power within the household will either endanger or solidify their ability

to act in order to effect positive change for health. This correlates with their level of

education and literacy.

5. Natural Environmental Factors

Many areas are prone to natural disasters that have the ability to quickly wipe out gains in

health and development. Some geographic regions are more prone than others to disruptive

natural phenomena. The profile of vulnerabilities will vary from location to location. Some

areas are prone to drought affecting food security and nutrition; others to quicker onset

disasters causing massive service disruption and/or displacement of populations (e.g.,

earthquakes, tsunamis, hurricanes, flooding, etc.). Unfortunately, the poor are

disproportionately affected by natural disasters.

The ability of a project to influence these subcomponents directly or indirectly is greater for

the first subcomponents listed than for the last. Some will consider the ecological environment

part of ‘the world as it is’ and not deserving to be part of a project’s analytical model. But in

some contexts, projects will find reporting on vulnerability and national preparedness important

to draw attention to threats to the sustainability of local efforts. This can be a tool for

advocacy. There can also be ecological variables with more direct effects, such as whether

mosquito breeding sites are controlled or not.

[39]

Handout 22 Enabling Environment Indicators Subcomponent Index Data Source

a. Health Policy and Government Commitment to Health

Commitment to Health Center for Global Development http://www.cgdev.org/content/publications/detail/10016

b. Governance and Civil Stability

Voice and Accountability World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp

Political Stability and Absence of Violence

World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp

Government Effectiveness World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp

Regulatory Quality World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp

Rule of Law World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp

Corruption Perception Index Transparency International www.transparency.org/policy_research/surveys_indices/cpi

c. Strength of Civil Society

Civil Society Strength Indicator

Freedom House www.freedomhouse.org

Indicator of Civil Liberties and Political Rights

Freedom House www.freedomhouse.org

d. Human Development Human Development Index UNDP http://hdr.undp.org/en/

Human Poverty Index UNDP http://hdrstats.undp.org/indicators/18.html

Progress toward Millennium Development Goals (MDGs)

UNDP www.mdgmonitor.org

Gini Coefficient (Measure of Income Inequality)

World Development Indicators www.worldbank.org/data

Equitable Access to Education and Health

World Development Indicators www.worldbank.org/data

e. Gender Empowerment

Gender-Related Development Index

UNDP http://hdr.undp.org/en/statistics/indices/gdi_gem/

Gender Empowerment Measure

UNDP http://hdr.undp.org/en/statistics/indices/gdi_gem/

Share of women in political office

UNDP

f. Ecological Environment and Natural Factors

Environmental Risk Index United Nations Environment Programme (UNEP)—Environmental Vulnerability Index (EVI), especially the questions related to disasters): http://www.vulnerabilityindex.net/EVI_2005.htm

General/Cross-Cutting Quality of Life Index The Economist‘s Intelligence Unit www.economist.com/media/pdf/quality_of_life.pdf

[40]

Handout 23 Detailed Sustainability Assessment Plan

Component

Indicators (Monitoring& Evaluation)

Definition of Indicator Source of Data

Frequency of Collection

Partner/ Institution

Responsible Baseline

Value

Program Outcomes

Health Services

Org. Capacity for Service Delivery

Org. Capacity for Comm. Mobilization

Community Capacity

Enabling Environment

[41]

Handout 24 (2 pages) Creating Scores from Scales A 'Scale' is a type of measuring device or weighing machine. As we begin to think about Scales, give your answers to the following: For each of the items below, check if it is better to rank high on the Scale, or low:

Better high on scale Better low on scale The number of cockroaches in my house My results on the pre-test Our team score in football The number of children in my family My body weight in kilograms

What did you choose? When is it better to rank high? When is it better to rank low? When is there debate about which is better? Now repeat the exercise using these sustainability-related issues.

Better high on scale Better low on scale Proportion of births attended by skilled health professional Cases of malaria Average annual provider surplus Number of days without rain in the last year Number of co-ops functioning

What did you choose? When is it better to rank high? When is it better to rank low? When is there debate about which is better? Sometimes the progression from poor to good or from undesirable to desirable does not occur at an even rate. Consider the following: If we are ranking the percent of community health volunteers who have received a supervisory visit during the previous quarter, we can use an evenly graduated Scale. It would look something like this:

Measured Value

Strong 81 to 100%

Promising 61 to 80%

Intermediate 41 to 60%

Emerging 21 to 40%

Poor 0 to 20%

However, if we try to apply the same Scale for ranking diarrhea treatment for children, it would not be appropriate. If a community has 61% of its community health volunteers receiving supervisory visits each quarter, that is promising! But if only 61% of the same community's children are receiving any treatment for diarrhea, that is not very promising. It could perhaps be better described as intermediate. A more useful Scale might be something like this:

[42]

Measured Value

Strong 90 to 100%

Promising 75 to 89%

Intermediate 55 to 74%

Emerging 35 to 54%

Poor 0 to 34%

Each of these Scales can be plotted on a graph, much like the common growth chart. Plot the following two charts using the lowest number in the Scales above and draw a line to connect the dots. Strong Promising Intermediate Emerging Poor 0 10 20 30 40 50 60 70 80 90 100

Strong Promising Intermediate Emerging Poor 0 10 20 30 40 50 60 70 80 90 100

If you plotted the above graphs correctly, the line drawn will look different on each. The first graph should have a line that goes like this:

This is called a 'linear scale'. The second graph should have a line that goes like this: This is called a 'non-linear scale.'

Percent of community health volunteers receiving a supervisory

visit in the past quarter

Percent of community children who get treatment when they have

diarrhea

[43]

Handout 25 An Example Sustainability Dashboard

Questions for Small Groups: Baseline How would you describe this Project at Baseline? Which Components are strongest? Which Components need more attention? If all 6 Components are critically important to the Project’s sustainability, on which Component(s) should we focus more attention in this example according to the Baseline measurements? Final How did the Project improve at Final? Which Components experienced growth? Which Components did not experience as much growth? List 3 reasons why you think that some Components improved more significantly than other Components. 1) 2) 3) What do you think is most valuable about this type of graphical summary?

[44]

Handout 26 An Example Pro-Sustainability Results Framework

SO: Increased uptake of HIV/AIDS and TB prevention, care and

treatment services

IR1: Improved coverage of

comprehensive HIV and TB services

IR 1.1: Expanded HIV and TB counseling & testing

service outlets

IR 1.2: Expanded HIV/AIDS care &

treatment services

IR 1.3: Increased coverage of HIV prevention services

(PMTCT and MC)

IR2: Improved quality of HIV and TB

health services

IR 2.1: Improved health worker performance for key HIV and TB care and

treatment services

IR 2.2: Improved supervisory system for facility and community-

based health workers

IR 2.3: Improved laboratory testing

services

IR3: Improved capacity & viability of Provincial Health

Teams

IR 3.1: Improved leadership and

management of HIV/AIDS and TB programs

IR 3.2:Improved technical capacity of district health

teams

IR 3.3: Improved capacity to lead and support

quality improvement initiatives

IR4: Improved capacity & viability of local NGOs and

CBOs

IR 4.1: Improved capacity to provide community-

based care and support to PLHIV

IR 4.2: Establishment of functional referral system

IR 4.3: Establishment of strong partnership among

NGOs/ CBOs

IR5: Improved community

awareness & knowledge of HIV &

TB

IR 5.1: Increased knowledge of HIV prevention among

community members

IR 5.2: Improved attitudes towards PLHIV

IR 5.3: Improved awareness of HIV/AIDS

and TB services available

IR6: Improved policy environment for

MARP

IR6.1 Provincial and national policymakers

informed of best practices for targeting services for

MARPs

IR6.2: National MOH decision makers aware of

specialized needs of MARPs

Vision: All people living with HIV and TB will be able to live healthy and productive lives. They will access quality health services and information, in a timely manner and within a supportive

environment.

[45]

Handout 27 Results Framework Template

Strategic Objective

IR1

IR1.1

IR1.2

IR2

IR2.1

IR2.2

Local System Vision

[46]

Handout 28 Program Planning Matrix

Result Area Strategy Activities Time Frame Stakeholder/ Partner Responsible

IR:

Sub-IR 1:

Sub-IR 2:

Sub-IR 3:

[47]

Handout 29 (5 pages) Sustainability Framework Checklist

Instructions for Participants

PURPOSE OF CHECKLIST:

This checklist can be used to review program plans and processes in order to integrate

sustainability into the design, implementation and evaluation of programs. It can be used during

the early conception of a project, the review of a project proposal, during project planning and

during a review of a project.

INSTRUCTIONS FOR THE PLANNING EXERCISE:

Use the checklist to think through what items you think are important for the implementation

of sustainability into the design of your project and to help you determine what actions to

include in your action plan.

The checklist can also help you to think about things that the project may have already done,

whether it be implicitly or explicitly. For any items that you identify that the project has

already done, think about and consider if there is anything else that you may need to do to build

upon the work that you’ve already done.

I. System Situational Analysis

# √ Question Comments

I.1 You can list all the essential members of

the local system [stakeholders] that will

have to play a role for positive health outcomes to be sustained 5-10 years

after the project ends.

I.2 You have a plan for approaching or you have already involved the identified

stakeholders in your project planning.

I.3 You can list other local system stakeholders (“key facilitators” and “key

allies”) and you have a plan to form

alliances with them.

I.4 You have identified the key outside influencers outside the local system who

should provide support to health related activities (e.g., national MOH, UN, NGOs,

Government); and you can describe how they influence those in the local system.

I.5 You can list and you have a reasonable

understanding of the priorities and plans

of the key local system actors and the key outside influencing organizations.

[48]

II. Sustainability Scenario Planning

# √ Question Comments

II.1 Local stakeholders have developed a

coherent vision for achieving and maintaining positive health outcomes in

the appropriate beneficiary group(s). This vision is a consensus of local stakeholders

and is meaningful, coherent, and (eventually) achievable.

II.2 You have identified potential conflicts

between the local system and the

agendas of other key local or outside stakeholders. You have a plan to work

jointly with implementing partners to resolve these conflicts.

II.3 The sustainability scenario offers the

project opportunities to make a significant and measurable contribution toward the

realization of the vision.

II.4 You have begun identifying implementing

stakeholders of the long-term strategies in the sustainability scenario for any

components/elements that are outside the mandate of the project.

[49]

III. System Assessment

# √ Question Comments

III.1 Project partners and other stakeholders

have defined the components and sub-components of the Sustainability

Framework for their local system context to guide what will be measured in the

local system assessments.

III.2 Project partners and other stakeholders have identified indicators to measure the

components of the Sustainability

Framework that apply to their local system context (for the baseline

assessments).

III.3 Project partners and other stakeholders have discussed and negotiated how the

local system assessments will be carried out, including:

a) Identifying existing data sources

b) Identifying and/or developing data collection tools

c) Determining roles and responsibilities for carrying out

local assessments.

III.4 Project partners collect, analyze and interpret data from local system

assessments carried out.

III.5 The project has synthesized the data to

make it meaningful for project partners and other stakeholders.

III.6 Results from the local system assessment

are shared with project partners and other stakeholders to inform the selection

of strategies/activities that will move the local system closer towards its vision.

[50]

IV. Program Planning

# √ Question Comments

IV.1 Project partners develop a Results

Framework or Log Frame, that is integrated into the Local System’s

Sustainability Framework (i.e., the project does not necessarily work on all

components of the SF equally, but has

taken account of all of them and is consistent with them).

IV.2 Project partners identify strategies,

activities and transition outcomes that will lead the local system closer to achieving

the vision.

IV. 3 The identified strategies and activities are not the sole responsibility of the project,

but other stakeholders have been

identified to work on the components or elements that are outside the scope of

the project.

IV.4 Project partners share the Results Framework/Log Frame and detailed work

plan with all relevant stakeholders in the local system for feedback and input.

IV.5 Project partners and other stakeholders

identify and negotiate the roles and

responsibilities of all stakeholders for carrying out the identified

strategies/activities.

IV.6 Project partners review the Results Framework or Log Frame and the detailed

work plan (consists of strategies, activities and transition outcomes), to

ensure that it is coherent and that it

captures all the elements necessary for sustainability.

[51]

V. Program Monitoring and Evaluation

# √ Question Comments

V.1 The Results Framework or Log Frame is

used to inform the M&E plan for the program.

V.2 The project M&E plan includes a

monitoring plan that involves active stakeholder review of progress toward

sustainability in all components of the SF,

and not just the components (and sub-components) that are within the mandate

of the project.

V.3 Project evaluations have maintained a focus on the local system in addition to

the strict requirements of accountability toward the project donor.

V.4 Monitoring, evaluation, and sustainability

assessment know-how are being

developed among local stakeholders themselves. Local stakeholders

demonstrate the freedom and initiative to call for review steps.

V.5 The nature and involvement of

stakeholders is reviewed over time; adjustments are made to project

implementation to increase cohesion and

ownership within the local system.

V.6 The project has shown flexibility to

respond to changes, shifts in the

environment, and lessons learned.

[52]

Handout 30 Sustainability Planning Action Plan Template

Sustainability Program Design

Process Step Activities Point Person(s) Time Frame

System Situational Analysis

Sustainability Scenario Planning

System Assessment

Program Planning

Program Monitoring & Evaluation

[53]

Handout 31 Creating a Sustainability Dashboard in Excel Task #1: Enter the Indicators into the Index Computation spreadsheet a) In the software provided, open the Index Computation spreadsheet b) In each of the six worksheets labeled at the bottom of the page Component 1-6, type in the

Indicators you selected under the column labeled “List Indicators.” c) To the right of the column labeled “List Indicators,” there is a column labeled “Measured Value.”

Enter the corresponding data from each Indicator in this column. For example, the Component 3 worksheet would look like this:

d) Enter your data for every Indicator, in all six Components, into the “Measured Values” column. e) Why is it important to transform these Measured Values into Scores? Discuss the following:

what is the difference between a Measured Value in percentage (%) and a Score in points? Once this is clear, proceed to the next step.

Task #2: Transform Indicator values into computable Scores a) Return to the first Component in the Index Computation spreadsheet. b) Select the first Indicator. Does the Indicator follow a liner scale or a non-linear scale? c) Open the Transform spreadsheet. d) For the selected Indicator, choose the appropriate Status Scale as seen in the yellow Tables

below:

Status Scale 1: Non-linear, positive slope Indicator status: Poor Emerging Interm. Promising Strong

max .349 0.549 0.749 0.899 1

min 0 0.35 0.55 0.75 0.90

[54]

Status Scale 2: Non-liner, negative slope Indicator status: Poor Emerging Interm. Promising Strong

max 1 0.45 0.3 0.15 0.05

min 0.4501 0.3001 0.1501 0.0501 0

Status Scale 3: Linear, positive slope

Indicator status: Poor Emerging Interm. Promising Strong

max .199 .399 .599 .799 1

min 0 .20 .40 .60 .80

e) Once you choose the appropriate Status Scale, enter the “min” and “max” numbers into the

corresponding boxes on the Transform spreadsheet. f) Examine the Indicator Status figure at the bottom of the Transform spreadsheet and briefly

discuss whether it represents a reasonable picture of progress on this indicator.

g) Identify the other Indicators in Component 1 which progress on the same Status Scale. [Note:

Indicators from the same Component might not all fit the same pattern, although they often do.] h) Enter the listed Measured Values of all Indicators in Component 1 sharing the same Status Scale

(and only those sharing this Status Scale!) under column D of the Transform spreadsheet. i) Which Indicators do not fit this Status Scale? j) Read and copy the resulting Scores from Column L of the Transform spreadsheet onto a piece of

paper.

[55]

k) Type the Scores from Column L (Transform spreadsheet) into the “Score” column of the Index Computation spreadsheet.

l) Select an Indicator that has a different Status Scale than the first Indicator you chose in the

Component you are currently working on. m) Repeat steps d through k for all of the Indicators that share this same Status Scale for this

Component. n) Once you are finished creating Scores for Component 1, repeat steps d through m for

Components 2 and 5.

[56]

o) The Transform spreadsheet we just used is for Indicators measured on a QUANTITATIVE scale. A different approach is needed for Indicators measured on an ORDINAL scale. For example, Components 3 and 4 (the OCIs) are measured on a 1 – 5 ordinal scale.

p) Select the appropriate Status Scale for the Organizational Capacity Indicators:

Status Scale 1: Non-linear, positive slope

Indicator status: Poor Emerging Interm. Promising Strong

max 2.4 3.2 4 4.6 5

min 1 2.41 3.21 4.01 4.61

Status Scale 2: Linear, positive slope

Indicator status: Poor Emerging Interm. Promising Strong

max 1.79 2.59 3.39 4.19 5

min 1 1.80 2.6 3.4 4.2

q) In order to change the status Scale for an ordinal Indicator, you must also change the x-axis (or

Measured Value) of the Indicator Status figure at the bottom of the Transform spreadsheet. r) In the Transform spreadsheet, double-click on any number on the x-axis of the Indicator Status

figure. A Format Axis window will appear. Click on the Scale tab. Type in your Minimum measure (in our example we will use 1), your Maximum measure (in our example we will use 5), your Major unit (in our example we will use 0.5), and your Minor unit (in our example we will use 0.1). Be certain that none of the boxes under Auto are checked. Click OK.

s) Examine the NEW Indicator Status figure at the bottom of the Transform spreadsheet and briefly discuss whether it represents a reasonable picture of progress on this indicator. If it does not represent your Indicator, please change the Status Scale (in yellow) and the x-axis of the Indicator Status figure. (See Transform_OCI.xls for an example of steps o through q.)

t) For examples of other Scales used to transform Indicators into Scores, please open the Examples_CSSAindexes.xls spreadsheet.

Task #3: Compile Scores into Index values a) For this exercise we have pre-selected the simplest way to compile the Scores into an Index (the

average). b) The Index Computation spreadsheet is pre-set to provide an average Score, which is the Index.

For example, you can calculate the average of column C, rows 4 through 15, by typing in the following text into an empty cell: =AVERAGE(C4:C15)

[57]

c) Follow this step for all 6 Components. Task #4: Create a dashboard a) Go to the worksheet labeled “Map” (always in Index Computation spreadsheet). The Indices

have been automatically inserted in the table. A hexagonal radar diagram (dashboard) has automatically been produced to represent all 6 Indices.

b) Discuss your dashboard with your group. In which Components is your Project strong? In which

Components does your Project need to grow? List three strategies for improvement (from three different Components). Each group will present their dashboard to the other groups.