carolyn sullins, ph.d. ladel lewis, ph.d. candidate the kercher center for social research western...
TRANSCRIPT
CONSISTENT PROTOCOL, UNIQUE SITES: SEEKING CULTURAL COMPETENCE IN A
MULTISITE EVALUATION
Carolyn Sullins, Ph.D. Ladel Lewis, Ph.D. candidate
The Kercher Center for Social ResearchWestern Michigan University
National Study
Comprehensive Community Mental Health Services for Children and Their Families Program: “Systems of care.”
incorporates a broad, flexible array of effective services and supports for a defined, multi-system population that is organized into a coordinated network… is culturally and linguistically competent, builds meaningful partnerships with families and youth at service delivery, management and policy levels, and has supportive policy and management infrastructure. (Pires, Lazear, & Conlan, 2008).
Systems of Care
144 sites have been or are in the process of being evaluated. Each 1-2 year cohort: +/- 30 sites.
Each SoC has distinct: geographic location and scope (e.g.,
statewide, county wide, city-wide, tribal) Ages of the youth served Mental health issues facing the targeted youth Racial, ethnic, and cultural factors
Core Values
Child centered and family driven
Community based
Culturally competent
Common Variables
Presenting issues of the youth Youth’s level of functioning (strengths
and weaknesses) Family strengths and barriers Types of services family and youth are
receiving Satisfaction with services Cultural competence of services Youth and family input into services
Barriers to Evaluation Implementation
“One size fits all” battery of questions
HSIRB mandates re language on consent forms
Ensuring an adequate sample size
Ensuring retention in a mobile population
Barriers to Participation History of racist abuse by researchers (E.g.,
Tuskegee syphilis study) Misinterpretation of data, or no access to
results Sensitive or stigmatized topics even more
difficult Families overwhelmed or embarrassed
Local System of Care:“Kalamazoo Wraps”
9%
90%
Insurance Private InsPublic Insn=244
Caucasian: 49.1%(including White Latino/a: < 5%)
African-American/Multiracial: 50.9%
Ages 7-17
Diagnosed with a Severe Emotional Disturbance
Academic/Professional vs. Local Population Perspectives
Informed consent
Voluntary participation
Confidentiality and its exceptions
Evaluation Work Group
Parents, various social service workers, eval staff
Reviewed consent forms for clarity Gave opinions to HSIRB re child abuse
reporting Parent input re: communication among
clinicians, families, and interviewers Interpretation of data Reporting of results
Youth Group
We came to their group
Help re local language
Helped us make it more comfortable for participants
Info that later helped us interpret data
Academic Culture vs. Local Communities’ Culture
Does confidentiality mean… No interviews in public
places, even if that’s what participants request?
Kicking Grandma out of the room?
Pretending you don’t see participant in public?
Problems with Interviews Some found it emotionally draining Too long and redundant
Questionnaires with overlapping questions Categories of services – national vs. local
terms Keeping in touch with families every 6
months
Easing Stressful Processes “Evaluating System of
Care – not you” Yet up front about sensitive,
personal questions Non-judgmental attitude
for better rapport, retention, AND accuracy. Balance – we can’t act as
friends or counselors! If SoC not working, or not
working with all groups of people, we need to know.
Interviewers: Pay Attention – So Participants Will Too!
Offer breaks, gum, stress balls, etc. to participants
Coloring books, DVDs for young kids
Redundant questions from multiple surveys: propriety and accuracy trump methodological “letter of law.”
Keeping in Touch with Families
Family address tracking form
Birthday and holiday cards with coupons
Incentives for families to contact us
Annual dinner as a “thank you”
Results in bimonthly newsletter, website, and other venues
Recruitment/Retention Results
Recruitment Rates (N=224)
77.2%
22.8%
African Americans
Partcipants
Non-Partic-ipants 80.9
%
19.1%
Caucasians
Participants
Non-Partic-ipants
Chi Square (1, N=224) = .466, p = .495
6 Month Interviews (N = 145)
72.10
27.9%
African American
Partic-ipants
Non-Partic-ipants 77.9%
22.1%
Caucasian
Partic-ipants
Non-Partic-ipants
Chi Square= (1, N= 145) = .665, p = .415
12 Month Interviews (N=119)
63.2%
36.8%
African American
Partic-ipants
Non-Partic-ipants 69.4%
30.6%
Caucasian
Participants
Non-Partic-ipants
Chi Square= (1, N=119) = .511, p = .561
18 Month Interviews (N=91)
59.6%
40.4%
African American
Participants
Non-Partic-ipants
56.8%43.2%
Caucasian
Partic-ipants
Non-Partic-ipants
Chi Square= (1, N=91) = .071, p = .834
24 Month Interviews (N=52)
47.8%
52.2%
African American
Partic-ipants
Non-Partic-ipants
62.1%
37.9%
Caucasian
Participants
Non-Partic-ipants
Chi Square= (1, N=52) = 1.055 , p = .402
Conclusion
Fulfilling national evaluation requirements within a local context is a constant balancing act.
With enough input from local stakeholders at each stage, it can be achieved!
Questions? Comments? Please contact
For more info, please see also http://www.wmich.edu/sociology
/kzoowraps.html