Measuring the impact and outcomes of local financial

inclusion initiatives Sinéad Marron, Toynbee Hall

Outline of Talk

1. Where’s this project based?2. Why are we doing it? 3. Who are you working with?4. How did we develop the Tool?5. What exactly will it measure?6. What do we hope it will achieve?7. What will it look like when it’s finished?

Followed by time for questions and discussion...

Toynbee Hall

• Community-based organisation• Vision:

“to eradicate all forms of poverty”• Current Projects include:

– Free Legal Advice Centre– Youth work– Older-people’s services– ESOL– Work with street-prostitutes– Volunteering & employment– Community Engagement– Time-banking

Advice & FI @ Toynbee Hall

• Services Against Financial Exclusion (banking, 1:1 work etc.)• Debt advice (Capitalise operator)• MacMillan advice• City advice• East End Energy Fit

• Financial Well-being measurement• Transact• Money for Life• Financially Inclusive Tower Hamlets• Local research

Measuring Impact: 6 observations...

Excellence

• Organisations with rigorous impact measurement systems

Lack of expertise and resources

• Many organisations lacking the capacity, resources or expertise

No single language

• No means of valid comparison

Poor national data

• Don’t know what we’re comparing results with

Output based funding

• No reliable, independent way of demonstrating soft outcomes

Tailored snapshots

• e.g. Debt advisors looking at financial capability

...build on existing practice

...be an independent

way of measuring soft

outcomes

...prevent multiple wheel

inventions

...create the opportunity for holistic assessment

...create a language to talk about

impact

...create a national database

Many of these observations came from local practice at Toynbee Hall

MAP Tool

Money, Access and Participation Tool

AIM: Develop a national financial well-being

measurement tool

ETHOS: Stakeholder led design process (by sector, for

sector) facilitated by independent experts

Partners and Funders

• Citi Foundation have funded a 2 year development and roll-out phase (June 2011 – May 2013)

• Partnered with National Centre for Social Research (Questionnaire Testing and Development Hub)

• Transact

Steering Group• Bristol University (PFRC)• Centre for Responsible Credit• Centrepoint• Citizens Advice• Department for Work and Pensions• Financial Inclusion Centre• Friends Provident Foundation• Hyde Housing Group• Liverpool John Moores University (RUFI)• Money Advice Service• Money Advice Trust• National Audit Office• Northern Rock Foundation• Southern Housing Group• University of Birmingham (CHASM)

Pilot Organisations• 2Shires Credit Union (PS1)• Fabrick Housing Group (PS1)• Bromley by Bow Centre (PS1 & PS2)• CHS Group (PS1 & PS2)• Civil Service Benevolent Fund (PS1 &

PS2)• Coastline Housing (PS1 & PS2)• Hyde Housing Group (PS1 & PS2)• Knowsley Housing Trust (PS1 & PS2)• Regenda Ltd (PS1 & PS2)• The Hillcrest Group (PS1 & PS2)• Wolverhampton Homes (PS1 & PS2)• Wrexham County Council (PS1 & PS2)• Advice NI (PS2)• City Save Credit Union (PS2)

• Community Housing Cymru (PS2)• Housing 21 (PS2)• Kirklees Citizens Advice Bureau (PS2)• London Borough of Camden (PS2)• Money Advice and Community Support

(PS2)• NHS Dumfries and Galloway (PS2)• North Liverpool Citizens Advice Bureau

(PS2)• Sandwell Citizens Advice Bureau (PS2)• Sandwell Homes (PS2)• South Yorkshire Credit Union (PS2)• Wales & West Housing Association (PS2)• Waltham Forest Community Credit Union

(PS2)• Women’s Employment Enterprise and

Training Unit (PS2)

Development Plan

1. Steering group2. Analysis of current impact evaluation3. Stakeholder consultation4. Develop and test initial questions5. Pilot 1 – 5 weeks6. Analysis & re-design7. Pilot 2 – 6 months with ongoing revisions8. Further consultations (including weighting)9. Analysis and redesign10. Move to web-based platform

What is financial well-being?

Building componentsInformation on what financial products and services the individual is currently using and how they are using them. Useful for identifying gaps and addressing issues of inclusion.

FINANCIAL PRODUCTS AND SERVICES

Bank Account

Type of Account

Bank Account Refusal

ID/Address Verification

Debit Card

Overdraft

Direct Debits

Savings

How Saving

Not Saving

Reasons for Not Saving

Insurance

Retirement Provision

Digital Inclusion

Access to the internet

Confidence/ability using the internet

9 Component Areas

• Demographics• Financial Products and Services• Income and Expenditure• Debt• Capability • Resilience• Attitudes• Well-being• Goals

• Vulnerability (composite area)• Barriers (flagged throughout)

Purpose and scope

Nationally

Sector

Funder

Organisation

Client

• Policy

• Lobbying • Sharing

information

• Impact & demand

• Impact measurement

• Needs assessment

• Trigger to find help, make changes and keep on track

What’s it actually going to look like?

•Web-based access

•Questionnaire format

The Tool itself

•Components are flexible

•Adaptable for different projects

Flexibility

•On paper, online

•Needs assessment output

Completing the Tool

•Revised needs assessment

•Summary of distance travelled

Repeat completions

•Aggregated individual impact

•Facility to review and manipulate data

Impact assessment

•Facility to share and compare data

•Anonymised national dataset (tool and survey data)

Data sharing

Any Questions?

Potential discussion points...

• Opportunities of the MAP Tool?• Risks of the MAP Tool?• Should a client be able to take their record

from one organisation to another?• What would sharing data allow organisations

to do?• What are the potential uses of the national

data?

Thank you!

We always welcome comments, thoughts or suggestions or questions.

Email me at:sinead.marron@toynbeehall.org.uk

Let me know if you want to be put on our mailing list or join Transact for regular updates.