Just a Click Away Conference, February 2011 Slide 1

Focus Consultantsfor the

Research & Statistics Division Department of Justice

Does PLEI Make a Real Difference to

Canadian Individuals, Organizations and Society?

Just a Click Away Conference, February 2011 Slide 2

In the current economy funders often requireempirical evidence that outcomes have beenachieved

PLEI providers, researchers and government policy

staff acknowledged that existing information and

research does not demonstrate the extent to which

PLEI has an impact on CanadiansJust a Click Away Conference, February 2011 Slide 3

1. Determine the degree to which outcomes andbenefits are addressed in current evaluationstudies and research

2. Identify the gaps, limitations and best practices in the research

3. Describe the barriers experienced byorganizations when doing outcomes/benefit

research

Just a Click Away Conference, February 2011 Slide 4

1. Review of PLEI evaluation research [2005+]

2. Interviews with PLEI providers and researchers (key experts)

3. Collection of and publication of vignettesthat demonstrate benefits and best

practices

Just a Click Away Conference, February 2011 Slide 5

1. Program/project evaluations

2. User assessments of PLEI products and resources

[web-based and other]

3. PLEI training assessments

4. Website statistics/Google Analytics reports

5. Client/provider satisfaction with PLEI

6. Distribution of and access to PLEIproducts/information

Just a Click Away Conference, February 2011 Slide 6

Identifying and Analyzing the Objectives:1. Inputs: Development and Capacity

Building

2. Outputs: Services/Products and Activities Delivered and Used

3. Impacts, outcomes and benefits

[individuals,organizations, society]

4. Client Satisfaction

Just a Click Away Conference, February 2011 Slide 7

Product and website developmentWebsite designImprovements to products and websiteTraining development and assessmentPartnership/intermediary training and supportPLEI delivery/access mechanismsStaffing, funding, structure, evaluation

Just a Click Away Conference, February 2011 Slide 8

Volume of clients served [including intermediaries]

Types of PLEI products or information

distributedNumber of PLEI activities heldLevel of use of productsDistribution networks created

Just a Click Away Conference, February 2011 Slide 9

User and intermediary group satisfaction with products, information outreach and services Satisfaction with website/design, visibility, usability

Just a Click Away Conference, February 2011 Slide 10

1. Increased knowledge exercising of rights2. Early prevention of legal problems3. More realistic use of and involvement with

thejustice system

4. Increased confidence/security 5. Resolution of legal problems6. Prevention of other problems (e.g. family

breakdown, homelessness, poverty)

Just a Click Away Conference, February 2011 Slide 11

Increased service efficiency/effectivenessBetter use of justice system by clientsMore helpful referrals

Just a Click Away Conference, February 2011 Slide 12

Most PLEI research focuses on short-termevaluation of projects and programs

Focus is on measuring access, volume, distribution, quality and satisfaction

There is limited attention to end-user applications of PLEI, outcomes and benefits

Just a Click Away Conference, February 2011 Slide 13

PLEI is complex (multiple topics, audiences,delivery mechanisms)

There are problems identifying/engaging end usersLack of follow-up to ensure high response ratesLack of capacity to assess benefits over the

long term

Just a Click Away Conference, February 2011 Slide 14

Limited use of methods to measure outcomesLack of benchmarks for web-based dataLack of evaluation expertise and skills (e.g. question

construction)Client confidentiality and privacy issuesProject/funder priorities and interestsLack of analysis of qualitative data

Just a Click Away Conference, February 2011 Slide 15

Inputs Outputs

Satisfaction(Intermediaries and

End-Users)

Outcomes/Impacts

and BenefitsTo

End-User

Just a Click Away Conference, February 2011 Slide 16