georgetown university national ta center for children's...

Georgetown University National TA Center

for Children's Mental 20th Annual RTC

Conference

Presented in Tampa, March 2007

1

Transformation Facilitation:Developing and Evaluating a Model

For Intensive TA For States

Sybil K Goldman, MSWSybil K Goldman, MSWRachele C Espiritu, PhDRachele C Espiritu, PhD

LanLan T Le, MPA T Le, MPANational Technical Assistance CenterNational Technical Assistance Center

for Childrenfor Children’’s Mental Healths Mental HealthGeorgetown UniversityGeorgetown University

March 6, 2007March 6, 2007

Overview and PurposeOverview and Purpose

Description of TransformationFacilitation initiative

Overview of evaluation approach

Preliminary results

Challenges in evaluating intensiveTA

FeedbackFeedback

Transformation Facilitation:An Overview

VisionVision

Supporting leadersSupporting leadersin states andin states andterritories to achieveterritories to achievetransformation oftransformation oftheir mental healththeir mental healthcare systems forcare systems forchildren and theirchildren and theirfamiliesfamilies

TF Background

Unique model that encompasses a varietyof approaches (coaching, TA, knowledgeapplication (2005))

Individualized approach to supportingleaders in states and territories in theirtransformation processes

Response to CYF Division request forongoing, intensive support

Partnership between Georgetown,National Association of State MentalHealth Program Directors, and UnitedAdvocates for Children of California

Overview of TF

TF faculty team assigned to eachstate and territory

Entrée to state through the statedirector of children’s mental health

Individual, peer, and expertcomponents

Multiple modes of communication



Conference


2

Overview of TF

Management team supports internaloperations

– Protocols developed for first threecalls

– Tools developed to support process

Monthly meeting of TF providers toprovide on-going training andsupport and to identify commonchallenge areas

Evaluation component

Family Involvement in TF

United Advocates for Children ofCalifornia (the Statewide FamilyNetwork Technical AssistanceCenter)

Membership on the TF Team

Active participation and input via TFMonthly meetings

States/Territory involved in TF

Sybil GoldmanNeal HorenArizona

Conni WellsRachele EspirituVermont

Liz WaetzigJan McCarthyPennsylvania

Sybil GoldmanRoy PraschilNorth Carolina

Gary MacbethConni WellsNew Hampshire

David MillerEllen KagenMissouri

Roy PraschilSybil GoldmanMinnesota

Ellen KagenDavid MillerMaryland

Gary MacbethJoan DodgeIndiana

Roxane KaufmannLiz WaetzigFlorida

Rachele EspirituNeal HorenAmerica Samoa

TF PARTNERTF LEADSTATE/TERRITORY

Role of TF Team

Assess and identify

Help create an action plan and timeline

Facilitate strategic thinking about leader’s

activities and linkage to outcomes (assess

risk, leverage opportunities, explorepartnerships)

Provide process and content expertise

Continually assess change

Link to TA offerings & peer opportunities,

and content TA

Support ongoing evaluation of efforts

Characteristics of TF

Planful, relationship based,accountable, journey

Strategic

Flexible

Intensive

Individualized

Enhancing the knowledge andskills

TF Hats

Facilitator

Coach

Mediator

Connector

Technical Assistance Provider

Broker



Conference


3

TF Individual Work

Phase 1: Orientation and overview

Phase 2: Information gathering

Phase 3: Identifying and prioritizingchallenge areas

Phase 4: Strategizing anddoing the work

State Issue Areas

Ensuring forward momentum andprogress in transformation in aconstantly changing politicalenvironment

Obtaining meaningful family and providerengagement in the implementation ofan outcomes- and data-driven mentalhealth system

Transforming public mental healthservices, without new $$, using crossagency workforce developmentstrategies

State Issue Areas

Determining role in developing a strongfamily network, identifying keyactivities, and engaging families in thework

Engaging and empowering child andfamily teams as the primarydeterminants of services

Overcoming resistance and difficulty inmoving the mental health field toward apublic health model

TF EvaluationTF Evaluation

(Even if you can get a pig to fly,

it doesn’t count if you don’t measure it.)

Evaluation Activities

Phase

Pre-Project

Start-up

Implementation and ProjectModification

Maintenance and

Sustainability

Replication and Policy

W.K. Kellogg Foundation Evaluation Handbook (1998)



Conference


4

Evaluation Components

1. Process/Outputs

2. Utility and satisfaction

3. Impact

Preliminary ResultsPreliminary Results

Process/Outputs Objectives

Characteristics of TF states

To describe the TF sample

Implementation

To describe project andreasons for deviations

To document challenges andsuccesses in implementation

To monitor the extent of TAprovided

States/Territory involved in TF

Sybil GoldmanNeal HorenArizona

Conni WellsRachele EspirituVermont

Liz WaetzigJan McCarthyPennsylvania

Sybil GoldmanRoy PraschilNorth Carolina

Gary MacbethConni WellsNew Hampshire

David MillerEllen KagenMissouri

Roy PraschilSybil GoldmanMinnesota

Ellen KagenDavid MillerMaryland

Gary MacbethJoan DodgeIndiana

Roxane KaufmannLiz WaetzigFlorida

Rachele EspirituNeal HorenAmerica Samoa

TF PARTNERTF LEADSTATE/TERRITORY

States/Territory monthly contacts

2126State/Territory K

1-41812State/Territory B

1-71310State/Territory C

1-28.57State/Territory G

110.59State/Territory I

2-41611State/Territory A

243State/Territory E

13.29.5AVERAGE

State/Territory J

State/Territory H

State/Territory F

State/Territory D

STATE/TERRITORY

12416

11512

21210

112.59

Number

of people

Estimated

number of hrs

Number of

contacts

Process Implementation issues

Determining readiness

Developing the relationship

Doing the work

Handling transitions and continuity



Conference


5

Determining Readiness

Readiness factor in the state or territoryis critical [TF]

Early on, I was disappointed but now Isee it as an opportunity to collectthoughts, get feedback, logic model,supervisory experience [CD]

TF gives me credibility-I’m one of 10 states involvedin this national effort [CD]

Developing the relationship

Most important aspect of TF: Having anoutside mentor/consultant who didn'thave a preconceived agenda aboutwhere I needed to be or should be goinghas been wonderful [CD]

How you start is the lesson learned. TFis very much a relationship-basedjourney - very different from anexpert model [TF]

Start where your client is [TF]

Doing the work

Most important aspect of TF: Havingsomeone to discuss and brainstorm thathad no other competing agenda than to dojust that. [CD]

Have to be ready for chaos - the more youdo it, the more that you begin to recognizeit and know how to step back. [TF]

The national perspective hashelped – can use it to leverage thevoice for kid’s agenda. [CD]

Individual vs. team approach

Handling transition and continuity

Changes in States and CDs

Internal politics has a lot todo with how the processworks [TF]

Have to be ready for chaos -the more you do it, the morethat you begin to recognize itand know how to step back.[TF]

Utility and Satisfaction

Usefulness – How useful is TF?

Satisfaction - How satisfied are CDswith the quality of TF?

Utility and Satisfaction:Peer Retreat Evaluation Results



Conference


6

Impact/Outcome Objectives:Individual

Short-term outcomes:Increase in knowledge, skills, and attitudesIncrease in leadership developmentIncrease in supportIncrease in knowledge of resources andconsultantsIncrease in peer sharing

Intermediate outcomes:Identifies deficits and transforms systemstructure and processes

Long-Term outcomes:Individual’s goals for system reform are realizedChildren and families get the services they needChildren and families live, work, and play in theircommunities

Impact/Outcome Objectives:System

Short-term outcomes:States are aware of availableresources

Intermediate outcomes:Local supports are developedLinks across federal grants andactivities

Long-term outcomes:Affect the national transformationagenda for children’s mental health

Impact/Outcome Objectives:National TA Center

Short-term outcomes:Ability to identify new resourcesIncrease in knowledge of statesIncrease in family involvement

Intermediate outcomes:Increase in Center’s ability to bemore strategic in planning TA

Long-term outcomes:Increase in sustainable partnershipswith states and communities toprovide TAAffect the national transformationagenda for children’s mental health

Impact/Outcome Objectives:Data Collection Instruments

CD baseline survey

TF baseline survey

Peer to peer retreat evaluationform

TF contact logs & progressnotes

CD priority area log

1 year follow-up survey & keyinformant interview with CDs

Yearly CD and TF follow-upsurvey

Impact/Outcomes:Knowledge Increase

Impact/Outcomes:Leadership Development

Baseline survey question: Supportleadership development– Mean = 3.4 (4 point scale)

Follow-up survey questions willexamine specific pieces of leadershipdevelopment:– Increased awareness and knowledge of

role as an agent of change

– Adopted new knowledge to lead change

– Implemented policies to support systemtransformation



Conference


7

Impact/Outcomes:Peer Sharing

Baseline survey question: Assist with thefacilitation of collaborative alliances– Mean = 2.7 (4 point scale)

Impact/Outcomes:Support

Challenges

Methodological

Contextual factors affectingresults

Next Steps

Implementation and ProjectModification– Development of follow-up survey

– Continued review of process data aboutsuccesses and challenges to modifyproject

– Collect and use short-termoutcome data to refineproject

– Share short-term findings with

stakeholders

Further down the road

Maintenance and Replication– Continue to use evaluation to

improve the project and tomonitor outcomes

– Assess long-term impact

and implementation lessons

– Share findings withstakeholders

– Determine critical elementsof the project which arenecessary for success

Gathering Feedback

What experience or ideas do youhave about evaluating this model?



Conference


8

Questions or Comments

Contact Information

Sybil Goldman• [email protected]

Rachele Espiritu• [email protected]

Lan Le• [email protected]

Appendix:Appendix:Orientation to theOrientation to the

Framework of TransformationFramework of Transformation

Defining Transformation

transformation \n (15c) From theLatin roots to change TRANS(across) and FORMA (shape)

1. a change in form, appearance,nature or character

2. the process of doing so

The New Freedom Commission describedtransformation as a vision, a process,

and an outcome.

Defining Transformation

A continuous and complexprocess

New behaviors, new competencies

New sources of power

New partners

Profound changes in structure,culture, policy and programs

Will not happen over night

SAMHSA/CMHS

Transformation as a Strategy forInnovation

ContinuousSmall Steps

A focus oncore missions,improvingwhat thesystem isalready doing.

A series ofManyExploratoryMedium Jumps

Pushing theboundaries of corecompetencies,trying to createsomething newwithin the existingparadigm.

A Few BigJumps

New rule setsthat leveragenew ideas.

SAMHSA/CMHS



Conference


9

Setting the Context for TF:Additional resources

Grounding our work

Setting the contextSetting the contextChange theory & frameworkChange theory & framework–– Tools for changeTools for change–– Dynamics of changeDynamics of change–– Leadership skillsLeadership skills

The Goals of aTransformation System

Goal 1 Americans understand that mentalhealth is essential to overall health

Goal 2 Mental health care is consumer andfamily driven

Goal 3 Disparities in mental health services are eliminatedGoal 4 Early mental health screening, assessment, and referral to services

are common practiceGoal 5 Excellent mental health care is

delivered and research is acceleratedGoal 6 Technology is used to access mental health care and information

Values and Principlesfor the System of Care

Comprehensive array of servicesIndividualized servicesLeast restrictive, most normativeenvironment that is clinicallyappropriateFamilies and surrogate families ofchildren are full participants in allaspectsIntegrated services with linkagesCase management

Stroul, B., & Friedman, R. (1986). A system of care for children and youth with severe emotional disturbances (Rev. ed.) Washington, DC:

Georgetown University Child Development Center, National Technical Assistance Center for Children's Mental Health. Reprinted by permission.

Values and Principlesfor the System of Care

Early identification and interventionfor childrenSmooth transitions to the adultservices systemProtected rights of childrenServices sensitive and responsive tocultural differences and specialneeds.

Stroul, B., & Friedman, R. (1986). A system of care for children and youth with severe emotional disturbances (Rev. ed.) Washington, DC:

Georgetown University Child Development Center, National Technical Assistance Center for Children's Mental Health. Reprinted by permission.

Change Theory & Framework

SOC is a Theory ofSOC is a Theory ofChangeChangeSAMHSA broadensSAMHSA broadensthe vision ofthe vision ofchangechange

georgetown university national ta center for children's...

Documents