More Resources, More Impact: PGP Evaluation Project

Presenters:Sustain Ontario – Phil Groff

Eco-Ethonomics Inc. – Ryan Turnbull, Devon Gregory, Trevor BensonFoodShare Toronto – Angela ElzingaCheng, Debbie Field, Yara Janes

Some opening remarks from:

Phil Groff, Executive Director

WELCOME

“

”

I practice and write about evaluation because I believe that evaluative thinking can make more effective those who are deeply committed to and authentically engaged in making the world a better place. Evaluation, at its best, distinguishes what works from what doesn’t, and helps separate effective change makers from resource wasters, boastful charlatans, incompetent meddlers, and corrupt self-servers. Michael Quinn Patton

Utilization Focused Evaluation

Evaluation has at its root, the word value

“

”

Utilization-focused evaluation is evaluation done for and with specific intended primary users for specific, intended uses. Utilization-focused evaluation begins with the premise that evaluations should be judged by their utility and actual use; therefore, evaluators should facilitate the evaluation process and design any evaluation with careful consideration for how everything that is done, from beginning to end, will affect use.Michael Quinn Patton

Utilization Focused Evaluation

Useful evaluation requires stakeholder direction

AGENDA, OBJECTIVES, AND PROJECT OVERVIEWPresenter: RYAN TURNBULL

Agenda

1. Welcome and Introductions2. Agenda and Objectives 3. Provincial Partnership Evaluation Project

Overview 4. Financial Mobilization Scan 5. Resource Sharing Library 6. Collective Impact Map 7. Cross Program Tracking Tool 8. Concluding Remarks and Next Steps

Webinar Objectives

1. To present each of the deliverables of the PGP Evaluation Project

2. To discuss how each of the deliverables can be utilized by other organizations

3. To reflect on the process thus far, and how the work will move forward

Provincial Partnership Evaluation Project

Overview• Project objective: develop replicable and

accessible tools for food organizations in Ontario to undertake their own evaluations

• What was developed:– Financial mobilization scan– Resource sharing library– Collective Impact Map– Cross program tracking tool

Provincial Partnership Evaluation Project

OverviewHow it was developed:•26 in-depth key informant interviews

– 9 with funders – 17 grant recipients

•3 online surveys– 1 for the financial mobilization scan– 2 for the Collective Impact Map

•2 in-person sessions– Sharpening the Tools for Change: Mobilizing Food System

Stakeholders Around Creating Greater Collective Impact– Choosing The Tools for Change: Mobilizing Food System Stakeholders

Around Creating Greater Collective Impact•3 webinars

– From Bland to Delicious: Spicing Up Evaluation– Mapping Collective Impact for Ontario’s Food Movement– More Resources, More Impact: PGP Evaluation Project

Project Process Diagram

Choosing the Tools for

Change

Financial Mobilization

Report

Online Survey

Evaluation Tool Review

Interviews

Start-up

Tool Harvesting

Sharpening the Tools

for Change

Tool DevelopmentWebinar

Financial Mobilization Scan and Impact Mapping

Food System Evaluation and Tool DevelopmentAnnotated

Tool Inventory

Community Partner Beta

Testing

Both

FINANCIAL MOBILIZATION SCANPresenter: Devon Gregory

Online Survey Demographics• Eligibility: Organizations engaged in food system

work in Ontario who had applied and/or received funding for food projects within the province

• 32 organizations completed the survey– 15 non-profits, 14 non-profits with charitable status, 1

grass roots committee, 1 unincorporated• Organizations varied considerably in size– Range from 0 to 80 full-time employees (average of 5)– Annual budget ranged from $5,000 – $4,300,000

(average of $487,518)

13

Applying for a Grant• Grants applied for on an annual basis = 9– Food grants applied for on an annual basis = 6

• Considerable time spent on applying and receiving a grant– 10 days spent applying– 100 days spent waiting to hear back– 69 days spent waiting for the money

• On average, 90 days are spent applying for grants (that’s three months of the year!)

Applying for Food Funding

14

Success rates when applying for food funding varied considerably amongst those who were surveyed. A large portion were rarely successful and another large portion were very successful.

Receiving Food Funding

• Average value of grants received ranged from $3,500 to $600,000 (average of all respondents was $174,524)

• Maximum amount of a food grant ranged from $4,000 to $989,000 (average of $162,747)

• Most prominent target populations: – Children and youth (52%)– Families (52%)– Low-income or marginally employed (48%)

Length of Food Grant Period

Areas of Food System Work

Organizations worked mostly in the following areas of the food system:•Food education (54.2%)•Food marketing (45.8%)•Food production (41.7%)•Social enterprise development (41.7%).

Funded Areas of Food System WorkMost of the work being done in the following areas is funded (>85% of projects in these areas are funded):•Food processing and manufacturing•Food marketing•Food education•Workforce development•Retail food outlets

Lack of funding being allocated towards other areas of the food system including: •Regulation and policy (only 22.2% of projects are funded)•Health and nutritional quality of food (only 57.1% of projects are funded)•Food access and nutrition (only 60% of projects are funded)

DISCUSSION

Resource sharing libraryPresenter: phil grofF

Scan of Evaluation Tools and Resources

Purpose:

To identify, collect, and organize for review, a breadth of evaluation tools and resources that are relevant to community food initiatives in Ontario.

Steps:

1.Website and Open Source Data Collection (online scan)2.Direct Data Collection (phone and survey)3.Website and Open Source Round Two (snowballing from survey contacts)

Results• More than 200 resources were collected,

scanned and 167 resources selected as a useful tools/resources

• Classification Scheme Developed (18 Categories)• Abstraction of Outcomes for the Collective

Impact Map• Consent received from resource owners to

share these publicly (with 6 still pending)

Where will these resources live?

• Working with Hypenotic to develop a resource sharing library as part of our Municipal Regional Food Policy Network project

• Evaluation Project will be one initiative on this library site

• Mix of display / offering based on different levels of consent:– Some hosted on Sustain Ontario’s site / FoodShare’s

site– Some abstracts with links to originating organization’s

site– Some abstracts without links

• Current plan is to launch for Local Food Week (June 6-12)

DISCUSSION

Collective impact mapPresenter: trevor benson

DISCUSSION

CROSS PROGRAM TRACKING TOOLPresenters: Angela elzingacheng and yara janes

Updated FoodShare

Tracking Tool

Updated FoodShare

Tracking Tool

Updated FoodShare

Tracking Tool

Why this tool?

1. Provide clarity and consistency2. It will align our data3. Deeper understanding of the intensity of a

particular program activity (depth of impact).4. Strengthen our evaluation processes5. Inform which projects should be prioritized.

DISCUSSION

CONCLUDING remarks AND NEXT STEPSPresenter: debbie field

Thank you for joining us today!