J A M E S S A N D M A N , L E G A L S E R V I C E S C O R P O R A T I O N D R . S A N J E E V K H A G R A M , I N N O V A T I O N S F O R S C A L I N G I M P A C T

D A V I D B O N B R I G H T , K E Y S T O N E A C C O U N T A B I L I T Y

2

I. Project Goals

II. Project Overview

III. Preliminary Survey Findings

IV. Next Steps

3

The goals of the project are to: • Enhance LSC’s ability to assess the quality, efficiency, and

effectiveness of the programs that LSC funds; and • Provide grantees with better tools to assess their own

performance, manage their operations, and increase private financial support.

Funding provided by the Public Welfare Foundation (PWF)

4

Project Overview

5

Needs Challenges Practices

Advisory Group

In-depth Interviews

Large N Survey

Landscape Review

6

Discussion and feedback from NLADA

Draft report based on comprehensive analysis of survey Feedback from community Finalize report

7

Assemble toolkit Update LSC Reporting System

8

Preliminary Survey Findings

9

0

5

10

15

20

25

30

35

1: Veryunsatisfied

2 3: Neithersatisfied norunsatisfied

4 5: Verysatisfied

1: Very unsatisfied

3: Neither satisfied norunsatisfied

5: Very satisfied

10

10

0

10

20

30

40

50

60

70

80

90

Yes No

YesNo

11

0% 50% 100%

Client outcomes +4 months

Client well-being

Monetary or non-monetarybenefits beyond client

Win/Loss ratios

Monetary value of client benefits

Direct non-monetary benefits

Client satisfaction

% collected (for utility,please see next slide)

% Do no collect but wouldbe useful

% Do not collect but wouldnot be useful

12

0 50 100

Client well-being

Win/Loss ratios

Client outcomes +4 months

Client satisfaction

Monetary or non-monetary benefits beyondclient

Direct non-monetary benefits

Monetary value of client benefits

5: Very useful4: Useful

13

0% 50% 100%

Client well-being

Monetary or non-monetarybenefits beyond client

Clients responses to services

Monetary value of client benefits

Direct non-monetary benefits

Client satisfaction

% Collected (forutility, please seenext slide)

% Do not collect butwould be useful

% Do not collect butwould not be useful

14

0 50 100

Client well-being

Client satisfaction

Clients responses to services

Monetary or non-monetary benefits beyondclient

Direct non-monetary benefits

Monetary value of client benefits

5: Very useful4: Useful

15

Rank order of top five data types 1. Economic and monetary benefits 2. Other outcomes data 3. Client surveys including satisfaction surveys 4. CSR data 5. Number of clients served

16

Top five not collected in rank order 1. Long-term/follow up data 2. Client satisfaction surveys 3. Economic and monetary benefits 4. GIS Mapping 5. Community impact

17

0 50 100

Identifying most useful data

Usefulness of data

Data systems and technology

Possible mis-use of data

Technical resources and tools

Staff expertise

Staff buy-in

Cost-effectiveness to collect from clients

Administrative burdens and costs

5: Very significant4: Significant

18

0 20 40 60 80 100

Referrals

Other services narrative

Web analytics

Pro Se assistance

Community legal education

Other grantee activity reports

CSR case closing codes

Problem codes

Number of person served

5: Veryuseful

4: useful

19

0 20 40 60 80

LSC responses to Competitive GrantApplications

Grantee inputs to CSR/CSM visit reports

Competitive grant applications

LSC inputs to CSR/CSM visit reports

Data for GAR (minus CSR data)

Grantee inputs to PQV report

LSC inputs to PQV report

CSR data

5: Veryhelpful

4: Helpful

20

0 20 40 60 80

Efficacy of data for external stakeholders

Efficacy of data for LSC

Usefulness

Compliance audit burden

Hard to measure

Cost

Sensitivity to context

“One size fits all” approach

Administrative burdens

5: Verysignificant

4: Significant

21

0 20 40 60

LSC data reporting systems efficient

LSC responds usefully to GARs

LSC data reporting not burdensome

Understand how LSC uses my data

LSC program visits yield useful feedback

5: Stronglyagree

4: Agree

22

The survey indicates that too few grantees report that they are benefitting from interactions with LSC around major touch points like GARs and PQVs. • Only 15% of respondents agree or strongly agree that

LSC uses data from the GAR to provide useful feedback about program operations.

• 44% of respondents understand how LSC uses the data they submit to the organization.

• 50% of respondents agree that feedback from Program Quality Visits provides useful data.

23

Next Steps

24

Toolkit to support LSC grantees

Improvements in LSC reporting system

25

• Continue engaging with stakeholders

• Publish detailed analysis of all work to date • Recommend selected and high value-add

reporting measures

J A M E S S A N D M A N , L E G A L S E R V I C E S C O R P O R A T I O N D R . S A N J E E V K H A G R A M , I N N O V A T I O N S F O R S C A L I N G I M P A C T

D A V I D B O N B R I G H T , K E Y S T O N E A C C O U N T A B I L I T Y