evaluating the effectiveness of citizens review panels blake l. jones, msw, lcsw program coordinator...

Evaluating the Effectiveness of Citizens Review Panels

Blake L. Jones, MSW, LCSW

Program Coordinator

Kentucky Citizens Review Panels

bljone00@uky.edu

www.uky.edu/socialwork/trc

"Never doubt that a small group of thoughtful, committed citizens can change the world; indeed it is the

only thing that ever has."  

Margaret Meade

“Service is the rent we pay forliving. It is the very purpose of lifeand not something you do in your spare time."

~ Marian Wright Edelman

“This whole citizens review thing is nothing but a public relations ploy”

~Anonymous CRP member

“We’re not really helping kids. All we’re doing is just generating another report that CPS won’t use!”

~Survey Respondent

Objectives

National Update

Dissertation proposal

What are YOUR thoughts one measuring CRP effectiveness?

How are state’s using CRP’s?

Great Variability> Numbers (Alaska: 5 volunteers, New Mexico: several

hundred)

> Staff Support

> Tasks

Recruitment vs. Appointment

Examples of Models

Created new panels (KY, Tenn.) > contract with Universities, other governmental agencies

Using existing panels (I.e., Child fatality review boards, regional or county QA teams, Governor’s task force teams). This appears common.

Hybrid (create new panels, but coordinate with larger group of existing panels)

Some states—Maryland, for example--have long history of “citizen review panels.”

What are Panel’s Doing?

Working on legislation issues (I.e, dual track response)

Examining community collaboration with CPS (through surveys, focus groups)

Evaluating state budget cuts on social services

Case reviews (I.e, looking for family involvement in case planning)

Employee satisfaction

Mandated reporters Administrative tasks

(by-laws) CPS’ involvement with

immigrant populations CPS’ relationship with

schools, law enforcement, mental health providers

Policy and Procedure (I.e., exit interviews)

CRPs used in the CFSR process

Minnesota Alaska (informal) Kentucky Wyoming South Carolina California (citizens reading PIP) Maryland (maybe?) Oregon Others?

Is it Working? Mixed results

Citizens having difficulty in defining their role, staying on track (easier with foster care review boards)

Seem to have changes on the local level (I.e., examining local policy and procedure)

Difficulty in recruiting diverse membership

Budget considerations (some recommendations not financially feasible)

Citizens want more feedback

Comments

“There is very little feedback from CPS so these panels rightly feel like their work is unappreciated.”

“ . . . Some team members are concerned that (their report) becomes so much paper in some big building”

“Some members do not have the basic skills to research a problem, develop a plan, begin working on it and track its progress”

“It makes no sense to me that an ‘objective’ body that is supposed to be evaluating a government agency would be housed within that agency”

Evaluation of Citizen Boards

Mostly focused on “outcomes” (i.e., Litzelfelner and CASA research)

Most researchers have found positive results from citizens boards

Perceptions of outside reviewers generally positive, though some distrust

reviewers (Leashore)

“Unauthentic” vs “Authentic” participation

King, Felty & Susel (1998)

Unauthentic• Conflictual• Input sought “after

the fact”• Reactive• Citizen treated as

necessary evil• Mistrust

Authentic• Collaborative• Input sought before

decisions are made• Proactive• Citizen treated as

partner• Trust

Difficulties in doing traditional program evaluations on CRPs

Wide variability in groups

Hard to measure impact of CRPs on child welfare systems (need longitudinal study)

What are “outcomes”? (reports?)

Shifting child welfare priorities (federal and state)

Dissertation Study

States Involved

Wisconsin Florida Alaska New York Alabama Ohio Tennessee Maryland Georgia

New Hampshire New Mexico South Carolina Arkansas Wisconsin North Carolina Wyoming West Virginia Nevada Minnesota

CRP Members and CPS Staff surveyed

Looking for:

> timely access to info

> training

> chairperson

> frequent contact with liaison

> ability to impact policy decisions

> Feedback from CPS

Arnstein’s Ladder

Where does YOUR citizens review panel fall on the “ladder”

Hypothesis: there will be a significant difference

Delegated Power: Citizens on your panel have the needed power to actively change the child protective system Partnership: The child protection system allows citizens to share in decision making, but retains all the power to change things  Placation: The system just “tells you want they want you to hear” about child protection in your state  Consultation: The child welfare system consults with citizens but does not give them any power Informing: The child welfare administrators in your state engage in “one way” communication with citizens (i.e., by providing superficial answers or discouraging questions) Manipulation: The child welfare system uses the citizens review panels to push its own agenda 

Other Variables

Previous leadership/volunteer experience

Gender, ethnicity (“representative” of community?)

Content of report/response to report

Limitations

Studying perceptions only

Point in time study (need longitudinal study to assess organizational change)

Generalizable nationwide?

What are

YOUR

Thoughts?