Ho

sted

by:

Fu

nd

ed b

y:

Where has the housing sector got to in measuring its social impact and what

approaches are being taken?

David Mullins and Vanessa Wilkes, TSRC, University of Birmingham Housing providers: evidencing their social impact

HACT Round Table Discussion 10:00am - 1.45pm on 7 June 2012, London

Social Impact – A Continuing agenda for housing sector

• Investing for Social Purpose in C19– Remember Peabody, Guinness? – implicit social enterprise model

• Social Investors of the 1920s – COPEC & Miss Fenter’s CI (financial inclusion and youth diversionary) activities!

• 1960s Cathy Come Home – housing as social movement – homelessness and neighbourhood renewal

• 1997 Giddens/Blair Social investment state – HAs as Social Investment Agencies – ‘from housing plus-CI’

• 2002 In Business for Neighbourhoods – sector rebranding - CI symbol of independent social purpose

• 2000s Government promotion of social enterprise & SROI across third sector

• Never forget the impact of secure affordable homes!!

Social Impact measurement in housing sector – research evidence• 2008 First NHF Neighbourhood Audit – first picture of extent

of CI (actually quite marginal to housing investment & management in most organisations but £435 million invested across sector, £272m from own resources)

• 2010 TSA Study – Community investment performance management toolkit for housing organisations - no golden bullet – wide range of tools - make or buy?

• 2011 Second NHF Neighbourhood Audit – important high level indication of change - but still mainly inputs & outputs

• 2010-13 PhD Study – from inputs & outputs to outcomes and impacts - Understanding why and how housing associations measure the social impact of their community investment activities (supported by NHF and engaging with HACT)

TSA Study 2010• Anglo-Dutch trawl of approaches to measurement– 17 tools

identified to plan, manage & measure CI activities • G15 Roundtables – scope and plans for measurement• 8 case studies – 4 internal tools, 4 off-shelf• Approach influenced by scale, type & organisation of CI & level

(individual, project, programme, corporate, sector) • Towards Impact - Considerable interest in moving from

inputs/outputs to outcomes/impacts• No established practice - Adaptation and use of wide range of

tools – choices often a condition of grant funding• NHF Audit – led to common scoping & classification activities

across sector – but significant differences in range of activities and ambitions of different HAs

• Launch of Community Impact Tracker as sector tool – would this standardise – enable benchmarking?

TSA Study 2010– Approaches & Gaps

• Projects – main focus, moving to harder quantitative approach alongside case studies

• Programmes – external accountability to funders, some strategies & theming – common reporting. Some move to standardisation & KPIs

• Corporate Overview – weakly developed – CI not on balanced scorecards – BITC, SROI, Social Audit being explored by a few

• Collaborative planning – weakly developed – organisational measures a barrier to collaboration? –credit claiming – going it alone

• Area Based – not much progress- floor standards, neighbourhood profiles looking dated- difference between nationals & community based HAs

• Ex-ante – Dutch focus on planning and goal setting – independent SEs?

• Ex-post – English focus on monitoring – regulatory mindset?

• Toolkit – no single tool meets all the aims – distance travelled tools for individual impacts – project management tools– corporate & sector indicators – collaborative planning tools (such as Outcomes Arena)

‘need more than a good story now to

fund CI’.

Outcomes arena – settingpriorities together with partners

So what’s really changing?• ‘Fences coming down’ – need for self-steering (more of

a Dutch approach needed?)• CI mainstreamed

• Economic crisis – need to harness the local £• HAs as SEs and as incubators of community and tenant

based SEs• 2012-13 Welfare Reforms – urgency of financial

inclusion work • ASB – recognition of CI investment in ‘diversionary

activity’ (remember Miss Fenter) to include in cost benefit analysis of ASB responses (HouseMark)

from ‘CSR extra’ to ‘core business’

What Drives your Community Investment work?

What kind of CI – what kind of measurement• Society led – responsive & consultative (measures

set with residents and communities)• Partnership led – LSPs & community commissioning

(measures set with LA and community partners)• Strategy led – strategic themes set priorities =

synergies with core business - (measures set corporately)

• Market led – the commissioning game – Supply chains and all that)- often based on individuals rather than neighbourhoods

–(measures set by contracts)

“If its not in the contract we don’t do it”

“CI washes its face”

‘making sure people

enjoy the projects’.

Hact Survey Methodology

Size of housing association

Not started any formal measurement and looking around for tools

Fairly new to measuring and waiting to see what results the current tools give them

Currently measuring but aware that need to make the tools / indicators better

Have established measurement systems and are able to see the benefits

Medium 1

Medium-Large 3 1 1

Large 1 4 310,000-29,999 2 1 3 530,000-49,999 1 1 350,000+ 1 1 2 7 respondents 4 respondents 10 respondents 13 respondents

- 34 Respondents- Self selecting organisations and interviewees- Telephone interview in November 2011

The stage of measurement activity

Growing importance / drivers

External• Wider third sector interest• Shift from ‘monitoring’ to

‘impact’• High profile networks

– SROI– Inspiring Impact– Think tanks

• Economic climate• Funders• “Keeping up” with the sector

Internal• Accountability

– Tenants– Boards

• Validate social as well as economic value

• Growing importance and integration of community investment

• Increased desire to understand neighbourhoods …. and see if making a difference

“Prove we are making a difference”

“If it’s not measured,

it’s not done”

Approaches to Impact Measurement

• Wide and varied approaches … in tools and methodologies

• Externally developed tools include:• Advice Pro; Balance Scorecard; Business In The Community;

CITs; CP Tracker; CR Tracker; Lamplight; Social accounting; SROI; Views (formerly SPRS)

Internally35%

Externally41%

A mix of internal and external tools

9%

No formal tool used15%

Paper based systems SROI across 4 countries

Common Issues• Whilst doing:

– Resources– Skills– Understanding complex

methodologies or tools– Development of

outcomes measures– Development of financial

proxies– Confidence (or lack of) in

reporting results

• Whilst thinking about it:– No perfect off the shelf

answer– Different tools for different

types of projects• Too much choice .versus. • no knowledge of the options

– Waiting for the golden bullet– Drawing on external

resources, consultants, networks

– Inter HA discussion “A common problem”

Overlapping

“What does good look

like”

The Purpose of Impact Measurement

Why do it?• Accountability• Self evaluation• Using the data

– for learning– in bidding

• Layers of measurement– Impact of some or all

activities?– Impact as whole

organisation?

Caveats• Importance of marginal

work• Funders demands

– Use of data– What is useful?

• Is it always appropriate and useful?– To housing associations– To them

Lesson Learnt • Steep learning curve

– Build on what achieved, expand breadth and depth • Ambiguity in …

– Methodology – Proxy values (e.g. SROI database)– Assessment does not give a definite answer– But … opens up debate

• Tension between ‘doing’ and measuring– Expectation of partners involvement

• Manage expectations• Promotes a culture change• A shared problem

Are you intending to change your measurement tool in the next 12 months?Response Reason External tool

usersInternal tool users

No Happy with current tool

Waiting to see the success of the one we are currently using

8

1

0

Yes Looking around for alternative tool(s)

Further develop the current tool(s)

2

1

7

2

Don’t know 2 3 TOTAL 14 12

Total Respondents: 26 hact research

- Need to move towards measuring outcomes (rather than outputs) and social impact- Need to keep up to date with new tools and methodologies- Need to investigate the functionality of our current tools

Moving forward ??

• Overall Strategy– Demonstrating the economic as well as the social

contribution of RSLs (not just community investment)

– More Group structures adopting a joined-up approach

• Resources– The need for appropriate time and resources– Analytical skills– Specialisation of roles

“Chaotic progress”

Moving forward cont..

• Methodologies– An area which needs improving – Recognition that the complexity of some

approaches may not fit all organisations / social enterprises

– Development of joint indicators – Greater use of proxy indicators– More methodological project planning /theory of

change– Arena for ‘challenges’ within projects to be

addressed

Questions• Value of standardisation within housing sector v

common approaches cross-sector?• Does measurement inhibit or enable

collaboration?• How do aims of CI and delivery models (society

led, partnership led, strategy led, market led) affect approaches to measurement?

• Is social impact of HAs just about CI or about whole business impact? (where are the measures of social impact of secure affordable housing?)

More Questions • How do motivations affect type of impact

measurement (external v internal drivers)• Should we wait for ‘golden bullet’ to solve

problems at a sector level? • How can progress become less chaotic?• What support resources do different types of

HAs need? • What can we best do together?

Thanks

For further info on TSA study, HACT survey and PhD please contact us at TSRC:

• d.w.mullins@bham.ac.uk• vew930@bham.ac.uk