roadmap 4: technical assistance to promote service and system change

8/9/2019 Roadmap 4: Technical Assistance to Promote Service and System Change

1/17

Technical Assistance

to Promote Serviceand System Change

Karen A. Blase

November 2009

ROADMAPTOEFFECTIVE

INTERVENTION

PRACTICES

www.challengingbehavior.org

4


2/17

e reproduction of this document is encouraged. Permission to copy is not required.

is publication was produced by the Technical Assistance Center on Social Emotional Intervention for YoungChildren funded by the O ce of Special Education Programs, U. S. Department of Education (H326B070002).e views expressed in this document do not necessarily represent the positions or policies of the Department ofEducation. No o cial endorsement by the U.S. Department of Education of any product, commodity, service orenterprise mentioned in this publication is intended or should be inferred.

Suggested Citation:Blase, K. (2009). Technical Assistance to Promote Service and System Change. Roadmap to Efective Intervention

Practices #4. Tampa, Florida: University of South Florida, Technical Assistance Center on Social EmotionalIntervention for Young Children.

www.challengingbehavior.org


3/17

1

Roadmap to Efective Intervention Practices

Technical Assistance to Promote Service and

System ChangeKaren A. Blase, November 2009

INTRODUCTION

Making a dierence for young children with challengingbehaviors is vital to promoting their social, emotional, andacademic success. Eective early intervention and preventionare critical to generating improved outcomes throughout thechilds life. Increasingly, programs and services for young chil-dren are interested in promoting both success and inclusionin diverse environments (e.g., home, pre-school, child care,and community) for young children with, or at risk for, delaysor disabilities. ese two goals of creating positive outcomesand inclusion are aligned with the goals that families have fortheir children (Odom, 2000) and with the legal mandates andregulations promulgated at the Federal and State level (Ameri-cans with Disabilities Act of 1990; Individuals with Disabil-ities Education Act Amendments of 1997). But what is the

best way to move from interest to action? How can attitudes,knowledge, practice, and policy be changed to support thesegoals? How will collaborative endeavors and systems need tochange to create sustainable, eective programs and practices?And what is the role of technical assistance?

Technical assistance (TA) has long been a standard, over-arching strategy for assisting, states, agencies, family members,and practitioners with building capacity for service and systemchange initiatives. But, what do we mean when we use theterm, technical assistance? What outcomes are we attemptingto achieve and who is our audience? What do we know about

the technical assistance strategies that are likely to achieveparticular outcomes? How can technical assistance strategiespromote both practice and systems change?

e purpose of this TACSEI Roadmap document is to assist arange of stakeholders (e.g., early childhood service providers

parents, technical assistance providers) in understanding thetypes of TA that are most benecial for achieving particularpractice and systems outcomes. e paper will explore andhighlight TA strategies to initiate, implement, and sustaineective practice and systems change. With this informationstakeholders at multiple levels (e.g., practice, organizationcollaborative groups, state, Federal) will be better equipped toselect, promote, and provide TA that is aligned with improvingpractice, organizations, and systems to serve young children with or at risk of disabilities with challenging needs. content of this Roadmap is based on a broad literature relatedto practice, service, and systems change, data and informa-tion related to TA across a number of domains (e.g., specialeducation, general education, community prevention, aid fordeveloping countries), and data and best practices related toimplementation and scaling up of evidence-based practices.

HOW IS TECHNICAL ASSISTANCE DEFINED?

What do we mean when we use the term Technical Assistanceere is no generic dictionary denition of technical assistance

is document is part of the Roadmap to Efective Intervention Practicesseries of syntheses, intendedto provide summaries of existing evidence related to assessment and intervention for social-emotionalchallenges of young children. e purpose of the syntheses is to oer consumers (professionals, otherpractitioners, administrators, families, etc.) practical information in a useful, concise format and toprovide references to more complete descriptions of validated assessment and intervention practices.e syntheses are produced and disseminated by the Oce of Special Education Programs (OSEP)Technical Assistance Center on Social Emotional Intervention for Young Children (TACSEI).


4/17

2

(TA) nor are there commonly adopted denitions of TA ineducation or special education. In fact, when State Departmentsof Education were surveyed and asked if they had a formal statewide denition of technical assistance, only one-third of thestates reported having such a denition (Council of Chief StateSchool Ocers CCSSO, 2005). However, when looking acrossthe attempts to dene technical assistance across a range of enti-ties (e.g., the World Bank and International Monetary Fund,US Department of Education, Northwest Regional EducationLaboratory), some common dimensions can be identied.

e most straightforward and overarching function of TAnoted by Choudhury (2001) was the transfer of new knowl-edge along with new technology to others who do not knowabout it. Specic to education, the Northwest Regional Educa-tional Laboratory conceptualized technical assistance as thetimely provision of specialized advice and customized supportto resolve specic problems and increase clients capacity(Barton, 2004).

e 2005 state survey ndings from the CCSSO further eshed

out the types of problems and capacity that states viewed asthe focus for TA. First, states indicated that TA was provided toaccess resources on specic topics and challenges related to servingstudents with disabilities. e second most indicated set of func-tions was to assist districts with policy planning and programimplementation in special education. ird, TA functions were

viewed as supportingschools/districts with schoolimprovement plans, compli-ance reviews, and providingsupport to low-performing

schools and districts.

ere is increasing attentionto providing TA focusedon developing capacity andproducing demonstratedimpacts at multiple levels.Broadly, capacity devel-opment emphasizes theability to dene and meet

challenges in a sustainable manner (International MonetaryFund, 2002). One useful operational denition of capacity

development is the process by which individuals, organiza-tions, institutions and societies develop abilities (individuallyand collectively) to perform functions, solve problems and setand achieve objectives (Godfrey et al., 2002). Godfrey et al.(2002) highlight the multi-level work of capacity building TAby recommending attention to four interrelated dimensions:

the development of individual skills and the condi-tions to ensure that skills are used productively

the development of eective organizations withinwhich individuals can work;

strengthening of interrelationships among entities

the development of enabling environments foraddressing issues across societal sectors.

And they note that developing individual capacity, whilnecessary, is not sucient to create the capacity to identifysolve, and sustain solutions. ey also refer to the capacity toreplace TA functions as a result of developing capacity within

the organizations and institutions themselves the essence ofregeneration and sustainability.

MATCHING TA INTENSITY TO DESIREDOUTCOMES

Fixsen, Blase, Horner and Sugai (2009) make the distinctionbetween Basic TA and Intensive TA. Fixsen et al. (2009) articu-late the value of Basic TA as an ecient approach for creating

readiness for change and facilitating change by providing infor-mation and support (e.g., materials, summative documentsoverview workshops, tools). ey postulate that Basic TA strate-gies are most eective when the capacity to achieve change bythe recipients is within the current abilities and skill sets of thoseinvolved and when funding, policies, and the infrastructure arealready in place to support the new initiative or new way of workPrimarily, what is needed is timely, accurate, accessible informa-tion about the innovation, the what. After Basic TA servicescreate, provide, and promote access to up-to-date informationand resources about the what, then educators, practitionersand administrators are ableto use the information because they

currently possess the skills and abilities (e.g., current competen-cies used in a new way) in a context that is largely hospitable(e.g., facilitating policies, funding, acceptance of the innovation)Or if not largely hospitable, the changes required are minor anddo not create signicant disturbance in the system. While noarticulated in their brief, it also might be likely that such TA wilbe episodic and shorter-term in nature. Initiatives requiring BasicTA are frequently encountered in education settings.

In contrast to Basic TA, Fixsen et al. (2009), indicate that Intensive TA (ITA) is required when new knowledge, skills, and abil-ities are called for and changes will need to occur at multiplelevels to support and sustain the new ways of work. Recipients oIntensive TA need to learn the skills and develop competencies

...Basic TA strategies are most efective when thecapacity to achieve change by the recipients is

within the current abilities and skill sets o thoseinvolved and when unding, policies, and the

inrastructure are already in place to support thenew initiative or new way o work.

...capacity developmentis the process by

which individuals,

organizations, institutionsand societies developabilities (individually

and collectively) toperorm unctions, solveproblems and set and

achieve objectives


5/17

3

related to the new content (thewhat) and supported to imple-ment with delity (the how).In addition, funding, policies,procedures, and regulations will need to be modied toalign with and support the newpractices and programs. Suchinitiatives also may requirethinking quite dierently aboutproblems, solutions, and maychallenge current beliefs andassumptions about the problemat hand and the potential solutions (e.g., inclusion, meaningfulparent involvement, early screening for social and emotionalconcerns). Intensive TA is inclusive of all the elements of BasicTA but requires considerable planning, frequent communica-tion, on-site work, collaboration at multiple levels, coaching,and both process and outcome evaluation eorts at several levels(e.g., setting, organization, state) to build capacity and achieve

systemic change. Intensive TA is well aligned with the multi-levelwork of capacity building detailed by Godfrey et.al (2002). atis, an overarching outcome of Intensive TA is to ensure that theTA functions and strategies are embedded in the capacity of theorganizations and institutions themselves to ensure sustainabilityand continuous regeneration and improvement.

e six core features of Intensive TA are:

Clarityrelated to agreement about the needs, vision,desired changes; mutually claried roles and respon-sibilities among all partners; agreement about how to

create new structures, lines of communication, etc.;and clear understanding of the current context (e.g.,system strengths, policies, stressors).

Frequent communication with respect to on-site meet-ings and telephone or video conferencing to initiateand manage change. And frequent cycles of planning,execution, evaluation, and articulation of next steps tomove the work forward and solve problems

Intensityof collaborative work to plan, prepare,prompt and create opportunities for reection,planning the next phase of development,and specication of next steps together with on-site coaching, assessments ofprogress, the infusion of new informa-tion into the system(s).

Duration of the work may be 2to 5 years to achieve systemicchange and the TA provideris committed to buildingcapacity during the entireterm of engagement.

Integrity refers to the a focus on creating a morecoherent and eective system of services and supportsthrough comprehensive work with the whole systemand the use of data at multiple levels to informdecision-making

Accountabilityfor assuring that intended outcomesoccur; using challenges and feedback as opportunitiesto bring in new strategies, partners, and knowledge

to continue the work; measuring impact at multiplelevels with benets to children and their families athe core. e TA Provider behaves as though they are100% accountable for results while simultaneouslyunderstanding and acting to create eective, collab-orative eorts to achieve agreed-upon goals.

In summary, Technical Assistance is dened in anumber of ways. But it maybe most useful to anchorthe denition in relation

to the intended outcomes.Technical Assistance topromulgate awareness andencourage changes in atti-tudes will require dierentstrategies than TA neededto build capacity and createservice and system change.TA can and should change over time and across initiatives tomatch the desired outcomes, from Basic to Intensive TA andback again. is gradient from Basic TA to Intensive TA can

be a useful way to analyze what is required. e followingvisual may be helpful in thinking about this gradient of TAservices in relationship to desired outcomes.

Recipients o IntensiveTA need to learn the

skills and developcompetencies relatedto the new content(the what) and they

also supported toimplement withdelity (the how).

Increased Knowledge

Increased Access to Inormation

Changes in Attitudes

Systems Change

Capacity Building

Figure 1. Technical Assistance Pyramid

Intensive

TA

BlendedIntensive and Basic TA

Basic TA

Technical Assistance topromulgate awarenessand encourage changes

in attitudes will requirediferent strategiesthan TA needed tobuild capacity andcreate service and

system change.


6/17

4

FACTORS RELATED TO ALIGNING TASTRATEGIES WITH DESIRED OUTCOMES

It is important for stakeholders and TA providers alike toanalyze key factors related to the change initiative along withthe desired outcomes. Assessing such factors can help TA recip-ients and providers to identify the intensity of the TA strategies

that may be required to successfully achieve desired outcomes.In addition, consideration of these factors allows TA recipi-ents to better assess, select, and partner with TA services. Andthe analysis allows TA providers to assess and build their owncapacity relative to the type(s) of TA they need to provide.

Some of the factors to be considered when analyzing an initiativeand matching TA intensity to outcomes include the following:

Degree to which the focus of the initiative is primarilyon improving knowledge, increasing access to up-to-date information, and/or impacting attitudes.

For example, practitioners in a child care setting canknowledgeably discuss the Pyramid Model or administra-tors are aware o changes in IDEA laws and regulations

Degree to which skill sets needed are already in thecurrent repertoires of sta (e.g., practitioners, super-visors, administrators, coaches) and can reasonablybe expected to be used in a dierent context or innew combinations.

For example, child care staf members who already skill-ully demonstrate the ability to provide descriptive praise

are asked to do so or a new target behavior.

Degree to which behavior/practice changes by front-line practitioners are required to achieve outcomes(e.g., child care sta, home visitor, preschool teacher)

For example, practice and policy changes that are institutedto reduce exclusionary practices or expulsions o childrenwith challenging behavior or to promote measureableincreases in positive peer interactions may require changesin a host o ront-line practices to achieve those outcomes.

Degree to which behavior/practice changes arerequired at multiple levels (e.g., front-line practice,supervision, administrative practices).

For example, supervisors need to make time or observa-tion o staf, to provide written and verbal eedback, andto partner with staf to do proessional development plan-ning. Or staf, supervisors, and administrators all needto review individual child and group outcome data on aregular basis to make program adjustments.

Degree to which funding and policies are currentlyaligned to support the initiative

For example, current agency policy does not allow rontline staf timely access to unds (e.g., petty cash or quickturnaround in unding requests) needed to purchasesnacks, arrange or transportation to community recre-ation sites, or to purchase materials in a timely ashion.Or stang levels and volunteer involvement do not allow

or the one-to-one services needed by some children.

Degree to which organizational and/or state resourcesand supports are available to support both the direcservice (practitioners work with children and fami-lies) and the required infrastructure (e.g., trainingsupervision, data system use).

For example, travel unds and insurance are not availableso that staf can visit amilies in their homes. Or coachingpositions and unctions are not unded to improve stafcompetency and condence.

Degree to which the initiative interacts with ordisturbs the rest of the service system (e.g., changesin referral process, new assessments, partnershipsrefocused or re-formed, competition with existingservices, requires new collaborative structures).

For example, social and emotional assessments andscreening are adopted and as a result expose service gapsrequiring a realignment o current services and changeto the service array.

Degree to which the initiative is designed to createor requires systemic change (broad and deep) versusencouraging islands of excellence or pilot demon-strations of feasibility.

For example, statewide screening eforts are adopted orinclusion and embedded services are implemented acrossan entire region or range o services.

Degree to which the new way of work and thinkingis signicantly dierent from the current culture andthinking (e.g., inclusion, using data on a regular basismeaningful family engagement, cross-system collabo-

ration) and will require changing hearts and minds aswell as practice and policy (Heitz & Laurie, 1998).

For example, advisory boards agree not to meet unlessparents are at the table. Or data-based decision-makingis introduced into preschool settings. Or cross-systemcollaboration among health, mental health, and earlychildhood is required to create a needed service.

In summary, the type and scale of the initiative and degree towhich new skills and behaviors are required, combined with thedegree of change required in practice, infrastructure, systems


7/17

5

partnerships, funding, and policies will help determine thetype of TA needed for a successful partnership between the TAprovider and the partners requesting assistance. A TA Discus-sion Tool is provided at the end of this paper to help partnersdiscuss and identify TA strategies that are aligned with theoutcomes expected and that are robust enough to achieve thedesired results.

INCREASING KNOWLEDGE & CHANGINGATTITUDES THROUGH BASIC TA

As noted above, Basic TA provides timely, accessible, up todate information that is needed by and useful to recipients.Such information is particularly useful for improving knowl-edge and changing attitudes. Paul and Redman (1997) in theirreview of studies of the usefulness of print material in changinghealth-related knowledge, attitudes, and behavior, found thatsuch print material for patient education was more consis-tent in changing knowledge and attitudes than in changing

behavior.

Certainly knowledge and information, provided through avariety of mediums, are crucial to stimulating and preparing forchange at the individual practice, organizational, and systemslevels. And there is a wealth of knowledge and informationabout early childhood services, supports, frameworks, assess-ments, and intervention strategies for young children with,or at risk for, delays or disabilities (Dunlap et al., 2006; Fox,Dunlap, Hemmeter, Joseph, & Strain, 2003; Ringwalt, 2008;Hurth, Shaw, Izeman, Whaley, & Rogers, 1999). For example,eective dissemination can increase knowledge, awareness ofresources, as well as promote changes in attitudes related to theinclusion of children with special needs in many communitysettings (Dunlap et al., 2006; Fox et al., 2003; Ringwalt, 2008;Hurth et al., 1999). Such information is increasingly accessibleat multiple venues (conferences, workshops, training institutes)and through multiple media (e.g., webinars, podcasts, news-letters, blogs, journals) for diverse groups of stakeholders (e.g.,parents, educators, administrators, etc.).

Of course, such information may be disseminated more or lesseectively. e National Dissemination Center for Children

with Disabilities (2009) draws attention to three criticaelements related to eective dissemination: access, under-standability, and utilization. ey also review key principlesabstracted from the knowledge management and dissemina-tion literature that are associated with eective disseminationese overarching principles include:

Using sources of information that are viewed as credibleby the recipient

Promoting a meaningful exchange between theinformation provider and the intended user of theinformation, including collaborative problem solving

Employing a social-marketing approach that isoriented to the specic, intended audience

Including multiple social components to increase theengagement, motivation, and support for users asthey attempt to utilize or apply the information

Eective dissemination of information is necessary to generateinterest in and readiness for actual changes in practiceprograms and policy that can result in improved inclusion andsocial-emotional outcomes for young children. People cannotbe motivated or ready for new service approaches and waysof work if they have not had the opportunity to understand,learn about, and consider the implications of such changes onmany levels. In fact, there is a body of research and theorythat indicates that eortsto make changes at theindividual or organiza-tional level are likely to

be much more successful when individuals havethe opportunity to movethrough their own stagesof change and when activ-ities and types of conver-sations are matched tothe individuals stageof change (Prochaska,Prochaska, & Levesque,2001). Studies have found that people progress through vestages of change (Prochaska & DiClemente, 1983) from

Precontemplation, in which the individual is not consid-ering making a change, to Contemplation, or thinking aboutmaking or joining the change initiative, to Active Preparationfor the change, to Taking Action and then Maintaining thatchange over time. Basic Technical Assistance can help prepareindividuals to move through the earlier stages of change, andget ready to take action and join new initiatives by providinginformation, creating the opportunity to discuss the newways of work and the benets and risks of the change, and byproviding the opportunity to try out new approaches in thesafety of a training or orientation setting.

...the type and scale o the initiative and degreeto which new skills and behaviors are required,combined with the degree o change required inpractice, inrastructure, systems, partnerships,unding, and policies will help determine thetype o TA needed or a successul partnership

between the TA provider and the partnersrequesting assistance.

People cannot bemotivated or ready or

new service approachesand ways o work i

they have not had theopportunity to understand,learn about, and considerthe implications o suchchanges on many levels.


8/17

6

Basic TA may not be sucient for systemic practice change,sustaining new skills, and changing organizational andsystems supports. However, the information sharing andexploration opportunities provided through Basic TA arecertainly necessary to the success of such complex andsystemic change processes and cannot be overlooked. In fact,research that compared Stages of Change distributions acrossa range of health behaviors and populations, found that forall of those individuals who were in a pre-action stage, only20% were in Active Preparation while 80% were in Precon-templation or Contemplation and not ready to take action(Velicer, DiClemente, Prochaska, & Brandenburg, 1985;Velicer et al., 1995; Laforge, Velicer, Richmond, & Owen,1999). While these studies were related to individuals whowere considering making a change in their health behavior,Prochaska et a l. (2001) point out that in organizational changeif 80% of sta in the pre-action stages are in a Precontempla-tion or Contemplation Stage,pushing them to take actionis sure to be problematic.

Individuals need time, infor-mation, and activities to getready for change. Providingspecic information aboutthe proposed change, discus-sion of pros and cons, andengendering condence in thechange are critical to helpingsta begin to change. Basic TA strategies can help eectivelyjump start practice, organizational, and systems changeand reduce so-cal led resistance to change.

Such Basic TA is not exclusive to the Exploration Stage (Fixsenet al., 2005) of implementation but may need to be embeddedthroughout the all stages of implementation during IntensiveTA eorts as well. After all, stakeholders, leaders, practitio-ners, administrators, etc. continue to come and go throughoutthe life of any initiative. Renewing and regenerating support,investment, and buy-in are shared activities among TAproviders, stakeholders, and direct consumers of the TA. Andif capacity building is an overarching goal, then these func-tions related to informing, educating, and preparing people toaccept and participate in change need to be embedded in thesystems and supports for the initiative.

BUILDING CAPACITY THROUGHINTENSIVE TECHNICAL ASSISTANCE

As noted earlier, Intensive TA is required when new knowledge,skills, and abilities are needed for change to occur at multiplelevels. While quantitative research on the impact of TA is in anascent state, it is important to note that positive impact has

been documented in a number of qualitative and case studiesin multiple domains including education, medicine, and a rangeof community-based and community coalition initiatives. esestudies indicate that TA supports help community groups inattaining broad goals and addressing complex issues (e.g., programselection, sustainability, board cohesion) (Feinberg, GreenbergOsgood, Anderson & Babinski, 2002). Positive results related toTA eorts also have been documented in the development andfunctioning of community-based AIDS networks (Yin, Gwalt-nery, Hare, & Butler, 1999) and in the development of tobaccocontrol coalitions (Kegler, Steckler, Malek, & McLeroy, 1998).

In the eld of early childhood inclusion for children with disabilities, Winton and Catlett (2009) cited a number of benets forthe 8 states who received Technical Assistance as part of theNatural Allies project. e project provided TA support to assistin building statewide capacity to enhance personnel preparationprograms and further develop faculty who prepare practitio-ners who, in turn, are caring for and educating young childrenwith disabilities in natural environments and inclusive settings

Benets of TA ranged from improved individual capacity (e.g.increased knowledge and skills related to inclusion by those whoare providing professional development), to increased organiza-tional and system capacity (e.g., increased family involvementformation of a statewide higher education consortium, continuedsystem-wide use of materials to promote inclusion).

With increasing attention to the capacity building function oftechnical assistance, examining research related to both indi-vidual capacity building and organizational/system capacitybuilding, helps to illuminate some of the strategies, outcomesand challenges of providing Intensive TA with this focus. e

following section of this Brief reviews both individual capacitydevelopment and organizational/system capacity developmenstrategies and what we are learning about their impact.

INDIVIDUAL CAPACITY BUILDINGTHROUGH INTENSIVE TA

Technical Assistance has been provided to impact individualcapacity development. As noted above, such TA has sometimesfocused on broad professional development initiatives (Winton& Catlett, 2009) and at other times has been more targeted,focusing on embedded TA to directly impact and improveskills needed for promoting delity and quality services (e.g.through coaching) (Hemmeter & Fox, 2009).

Evaluation and research results that review the impact of TAfocused on individual capacity development (e.g., technical assis-tance in the form of coaching or feedback for practitioners) indicatethat such practice-specic TA is critical to improve practitionercompetence and ensure program delity (Joyce and Showers2002; Durlak, 1998; Kelly et al., 2000; Harchik, Sherman

Basic TA strategies

can help efectively jump start practice,organizational, andsystems change and

reduce so-calledresistance to change.


9/17

7

Sheldon, & Strouse, 1992). Program and practice delity refers tothe degree to which the innovation is implemented as intended byprogram developers and/or researchers in order to achieve positiveresults (Mowbray, Holter, Teague, & Bybee, 2003).

While attention to delity and its relationship to outcomes hasnot received as much attention in education as it has in otherdomains, particularly with regard to curriculum studies, thereis a current focus on the importance of delity in interpreting

results of both ecacy and eectiveness studies (ODonnell,2009). With studies in multiple domains indicating that higherdelity is correlated with better outcomes, this impact of TAto build individual capacity may be signicant in achievingmore consistent, positive results (ODonnell, 2009; Hopkins,Mauss, Kearney, & Weisheit, 1988; Mitchel, Hu, McDonnell,& Swisher, 1984; Tortu & Botvin, 1989).

TA eorts aimed at improving individual practitioner compe-tence will require Intensive TA strategies because improving andsustaining practitioner competency inevitably leads to the needfor organizational and systems change. Research and evalua-

tion ndings indicate thatsuch factors as organiza-tional culture and lead-ership, resources, labor-relations, scheduling, andthe need for participatoryplanning are variablesthat impact the eective-ness and sustainabilityof coaching in educa-tion and special educa-

tion (Joyce & Showers,2002; Denton, Vaughn,& Fletcher, 2003; Marks& Gersten, 1998). Suchorganizational and systems factors extend to other domains as well including the adoption of school-based tobacco preven-tion (McCormick & Brennan, 2001) and in the eld of mentalhealth (Kavanaugh et al., 2003).

In addition to Intensive TA to address such organizationaland systems factors, there is also the need for multi-level indi-vidual capacity developmentwithin agencies and/or the service

sector. at is, not only must the competency of practitionersbe developed and sustained, but the competency of coachesalso must be developed and sustained. is is especially crit-ical since quality coaching embedded in the service setting isassociated with practitioners actually using newly acquiredskills (Fixsen et al., 2005). erefore, developing or improvingcoaching may be an important focus of Intensive TA. Inten-sive TA goals include ensuring that systems are put in place todevelop, support, and monitor processes for selecting, training,coaching, and evaluating coaches as well as measuring thedelity of the coaches behavior to the intended coaching

process. Research indicates that Intensive TA eorts need tofocus on ensuring that coaches/consultants have the contentknowledge necessary to be helpful (Schoenwald, Sheidow, &Letourneau, 2004) and that adherence of consultants (coaches)to coaching protocols can be linked to subsequent levels otherapist adherence which in turn are linked to better therapistdelity and better outcomes for youth.

In summary, Intensive TA ensures that organizational and

systems that interact with individual competency (e.g., fundingtime, policies) are addressed and that systems are developed toimprove, sustain, and regenerate the competencies of practitio-ners and coaches. Such multi-level organization and systemsdevelopment has been a hallmark of School-Wide PositiveBehavior Interventions and Support (SWPBIS) (Barrett, Brad-shaw, Lewis-Palmer, 2008) ensuring that supports and capacityexist at school, district, and state levels; and similar attentionhas been paid to such multi-level eorts in promoting program-wide adoption of the Pyramid Model (Fox et al., 2003). Fullan(2007), Kahn et al. (2009) and Fixsen, Naoom, Blase, Friedman

& Wallace (2005), all make the case for more systematic, multi-level, multi-year approaches that align capacity on many levelsto support practice change and to create hospitable environ-ments that facilitate new practice implementation and sustain-ability. e characteristics of Intensive TA noted by Fixsen eal. are more likely to promote such multi-level alignment andto develop the supports needed to build individual capacity atmany levels in agencies and systems.

Summary of Research and Evaluation FindingsRelated to Individual Capacity Development

Fidelity is a measure of individual capacity to imple-ment an innovation as intended

Because high delity is consistently associated withimproved outcomes across many domains and studies,it is important to create and sustain practitioner compe-tence that results in high delity implementation

Coaching (e.g., observation, feedback, data reviews)has been demonstrated to be an important core imple-mentation component to improve individual capacityof practitioners

Adherence to coaching routines has been demon-strated to be associated with improved practitionerdelity, and higher practitioner delity has then beenassociated with improved outcomes for children

TA eorts aimed at improving individual practitionercompetence inevitably lead to organizational andsystems change factors that require multi-level, Inten-sive TA eorts

(continued)

TA eforts aimed atimproving individual

practitioner competencewill require IntensiveTA strategies because

improving and sustainingpractitioner competency

inevitably leads to the

need or organizationaland systems change.


10/17

8

Summary of Research and Evaluation... (continued)

Intensive TA is required because individual capacitymust be developed and sustained at multiple levels(e.g., practitioner, supervisor, coach)

Multi-level capacity development, even when focusedon individuals, requires change at the organizationand system level (e.g., new roles, functions, activities,policies, funding) to create hospitable environmentsfor new ways of work.

ORGANIZATION AND SYSTEMS CHANGE:INTENSIVE TA IMPLICATIONS

From a research perspective, very few quantitative evaluation orresearch studies are related to the impact of broad technical assis-tance endeavors intended to change not only practices but alsoorganizations and service systems. However, there are some keystudies in human services that have begun to provide quantita-tive data and process perspectives regarding TA impact throughan evaluation lens. is section of the paper will review threesuch studies that were selected because they report both processand outcome data and have implications for TA geared towardcapacity building through organization and systems change.

e rst study is in the eld of prevention and focuses on the useof community coalitions. Feinberg, Ridenour, and Greenberg(2008) examined the impact of on-site and o-site (email and

phone) TA provided by ve regional TA providers in Pennsyl-vania on the functioning of Communities at Care preventionboards. e boards were charged with initiating and developinga community-wide youth-development and prevention plan-ning eort to promote the positive development of children andyouth and prevent problem behaviors such as substance use,delinquency, teen pregnancy, school drop-out and violence.

e study looked at important factors to be considered in devel-oping TA systems to support board development including:

on-site vs. o-site contact

developmental phase of the project

level of functioning of the coalition boards.

Self-report data and ratings on board functioning were collectedfrom board members and TA providers over three years alongwith TA provider data from monthly reports on time and typeof TA and their assessment of need for TA for each site.

Some of the key ndings include the following. Overall, there wasminimal evidence that TA dosage was linked to improved coali-tion functioning. However, when moderators were considered,

the results may help us understand some of the conditionsunder which on-site TA, which is relatively expensive, may bemore eective. On-site TA had a modest and moderate posi-tive impact on board functioning for newer boards. For boardsthat were higher functioning at baseline, there was a modestto moderate positive eect of on-site TA. e authors postulatethat these results may be due to the fact that higher functioningboards (whether new or old) may be better prepared to botharticulate their needs and take advantage of TA, while lowerfunctioning boards may need time, attention, and TA intensitythat could not be provided in this eort.

In terms of intensity of TA, the mean number of minutes of TAper month ranged from about one-hour a month for o-site TAto over 2 hours a month for on-site TA. us, it is possible thathis level of intensity generally was not sucient, regardless ofthe type of TA, for low functioning boards to move forwardCongruent with this hypothesis was the nding that greatero-site TA dosage, in comparison to on-site dosage, was asso-ciated with lower levels of board functioning for boards with

high perceived needs.e study authors caution that the eects of TA dosage and typemay vary signicantly depending on the community modelbeing implemented as well as the characteristics of the TAproviders and that it is dicult to generalize the conclusions othis study to other models and community-based eorts. eyalso acknowledge that the quality of the TA eort was not parof the analysis. ey advocate for a more comprehensive evalu-ation approach to TA in which dosage, quality, focus, and needare all attended to and measured in relation to outcomes.

Kahn, Hurth, Kasprzak, Diefendorf, Goode, and Ring-walt (2009) also looked at TA strategies related to impactingsustainable systems change and building capacity, but with afocus on state early intervention and preschool special educa-tion programs under the early childhood provision of the Individual with Disabilities Education Act. is evaluation eortarticulates a TA model for systems change and provides ananalysis of the impact of utilizing that model as reected in 32to 37 state plans over a 5-year period.

e TA model hypothesized that multi-level inuences needto be identied and targeted for impact in order for outcomesfor children and their families to be improved. us, theirmodel recognizes that:

"state inrastructure, local inrastructure, and systemso personnel development interact either to support or tohinder the implementation o efective practices at thelocal level, which, in turn, afect the outcomes or chil-dren with disabilities and their amilies." (p. 27)

is translated to a TA model for systems change that includedsystematic eorts to attend to multi-level strategies and impactsand was grounded in frameworks that recognize that systems


11/17

9

components are interrelated, dynamic, and reciprocally inu-ential (Datnow & Stringeld, 2000; Fullan, 1999; Fullan,2007; Fixsen et al., 2005; Harbin, McWilliam, & Gallagher,2000; Hebbler & Wagner, 1998; Trohanis, 2004).

Process evaluation methodology captured variables related tohow the TA providers time was spent in relation to 12 activitycategories related to the clients stated purposes of the interac-tions with the TA provider (e.g., marketing, selecting the issue,

developing and rening the plan, tracking and promotingimplementation). ese data indicated that:

54% of the time was invested in up-front workrelated to selecting and clarifying the issue, planningthe process, developing the plan and reviewing thequality of plan

12.6% of the TA time was invested in implementa-tion related activity

30.4 % was invested in providing content specic TA.

ere was tremendous variability across states in relation toTA time devoted to each area reecting the individualizationof TA services and supports. e median total time for plan-ning and implementation across the 32 state plans was 255.75hours over a 1.5 to 3 year period. is is a considerably higherlevel of TA service than was documented in the Feinberg et al.(2008) study of TA for community boards and is more reec-tive of Intensive TA eorts.

ree key recommendations for determining state readiness forsystems change work emerged from the analysis of the processdata. e rst set of recommendations related to selecting aninitiative that would:

Be able to improve the states capacity to address theissue successfully

Make a signicant dierence for children and families

Possess a degree of urgency and/or consequences forfailing to act

Be clearly dened within a scope of work that couldbe addressed in a 1 to 3 year project.

e second recommendation related to factors for selectingstates and included:

Commitment of key leaders,

Readiness and ability to devote time and resources,

Coordination with current activities to tie initiativestogether and not duplicate eorts,

Meaningful involvement of key stakeholders.

e third recommendation involved ensuring adequate TAresources for implementing change. A TA team must be assem-bled and maintained with adequate time and expertise related to

Knowledge of the states current context

Planning and process expertise

Topical expertise

Expertise in facilitating collaborative work withother TA centers and experts.

In examination of plans for multi-system impact, plans that werefully implemented had greater results. e study reported thefollowing results, with percentages in parentheses representingthe nine plans that were fully implemented. With respect tomulti-system impact, the study reports that 97% (100%) of plansshowed improved state systems, 54% (67%) showed improvedlocal system infrastructure, 51% (67%) showed improved prac-tices for service providers, and 35% (44%) showed improvedresults for children and families. Increased results for fully implemented plans help make the case for full delity to the plan.

Improvement and continuation of work were two dependentvariables analyzed as outcomes. e results of these analysesindicated that:

A state work plan had slightly lower odds of continuingif the state initially requested a specic TA service andNECTAC (the TA entity) responded by suggestingthe systems change TA Model process as the strategy

Plans where TA sta took more of a lead, but no

total leadership, had better results. However, whenTA sta took all, or almost all, of the leadershipresults were lower.

ere was no signicant dierence in either improvement or continuation based on breadth of stakeholderepresentation.

More total TA time only slightly increased results andincreased interaction during implementation showedonly slightly higher results

Better results occurred when states consistently used

referred to, and updated their plan

Better results occurred when the state was activelyengaged with TA consultants for a sucient amountof time to develop a plan and to receive TA services

e Kahn et al. (2009) evaluation strategies and results point theway to the importance of TA eorts that are targeted at multiplelevels, account for readiness of the TA recipient, are guided bythoughtfully developed and frequently used plans, and thainvolve the TA Center as connecting hub for the work.


12/17

10

e third study provides quantitative evaluation data thatindicate that the provision of TA focused on organizationalcapacity building was associated with more successful imple-mentation eorts and with sustainability of service deliverycapacity (Fixsen & Blase, 1993; Fixsen, Blase, Timbers, &Wolf, 2001). ese data were collected over a 20-year periodlooking at both the longevity and quality of service providedby front-line practitioners utilizing the Teaching-Family Modelof treatment (i.e., Teaching-Parents in therapeutic, commu-nity-based homes) and the sustainability of organizations (e.g.,sites) that supported networks of group homes within a state.

Initial eorts at building capacity to implement the Teaching-Family Model were focused on building individual capacityof the practitioners and involved the provision of TA from asingle national location to Teaching-Parents who were oper-ating group homes across the country. e implementationchallenges and program level data soon led to the programdevelopers to conclude that,

efective, sustainable replication meant shiting rom anational dissemination strategy to a regional approach thatocused on the development o regional training sites thatwould in turn support networks o group homes (Fixsen& Blase, 1993).

is shift meant developing more geographically proximate orga-nizational capacity that could utilize Intensive TA eorts to buildboth the individual capacity of practitioners (e.g., select, train,coach in person, evaluate) and attend to systems change issues(e.g., licensing, funding streams, local board development). edata correlated with this shift indicated dramatically dierentsurvival rates for group home programs than were experiencedunder the national, centralized model. e rst 25 replications,under the national model, focused on practitioner developmentand 56% of those programs ended when the rst couple trainedleft that group home. Only 24% of those programs continued toexist after 6 years or more. In contrast, under the organizationalcapacity development model, only 4% ended after the originalcouple left, and 84% of the group home programs continued tooperate after 6 years; a dramatic increase in sustainability.

e second set of lessons learned from the TA eorts involvedthe survival of the regional sites that supported networks ofgroup homes in a proximate geographic area. e rst eortsto establish regional sites occurred when certied Teaching-Parents, who also were graduates of the doctoral or mastersdegree program, launched and staed these new regional sites.However, when degreed professional sta without such directpractice experience attempted to do so, the sites did not fareas well, as indicated by the decreased likelihood of the siteachieving site certication under the criteria established by theTeaching-Family Association and the reduced longevity of thesite (Fixsen & Blase, 1993). e improvement in quality (asindicated by achieving certication) and survival (6 years ormore) was associated with more formal organizational capacity

development through Intensive TA rather than total relianceon the professional training of the site sta. at is,

"sites became successul when we initiated an integratedsite development system to train the trainers, consultants,evaluators, and administrators; consult with the trainers,consultants, evaluators, and administrators; evaluate theperormance o the trainers, and provide acilitativeadministrative supportto a new site" (p. 607).

While the longevity and quality data from the replication oTeaching-Family model homes and sites are only correlatedwith the shift in TA strategies, they do indicate that TA focusedon purposefully developing organizational capacity may beimportant for achieving more systemic impact resulting inimproved service quality and sustainability.

Summary of Research and EvaluationFindings Related to TA and Organizationaland Systems Change

More research and evaluation eorts are needed to deter-mine the relevant dimensions of TA and its impact onchanging practice, organizations, and systems. Becausethe knowledge base represented here is not yet mature orrobust, the summary ndings are oered only as a begin-ning point for future investigation.

Data from the above studies do provide some indication that:

Multi-level TA eorts that attend to planning, infra-structure, partnerships, and individual, organiza-tional change, and systems capacity building may be

eective.

TA that develops both individual capacity at multiplelevels (e.g., practitioner, coach, administrator) and orga-nizational capacity (e.g., funding, policy) increases thesustainability and quality of system and service change

On-site TA may be more eective for entities that arehigher functioning and thus able to articulate theirneeds and take advantage of TA.

Lower functioning entities may require more time,attention, and intensity of service

Initiatives selected for organizational and systems changeare more likely to succeed when the initiative chosen: Improves the state/entitys capacity to address

the issue Makes a socially signicant dierence for the

end user (e.g., children, families) Possesses a degree of urgency for failing to act Can be addressed in the time allotted (e.g., 1 3

years)

(continued)


13/17

11

Summary of Research and Evaluation... (continued)

Selecting entities that are ready is important. Keycomponents of readiness include: Commitment of key leaders Ability to devote time and resources Coordination, not duplication, with current

initiatives

TA Resources need to be adequate including time,knowledge of the context, process and content exper-tise, and ability to collaborate with other entities

Geographically proximate TA may be more eectivethan TA from a single national entity

oughtfully developed and frequently used andupdated plans may be associated with better outcomes(e.g., quality, sustainability)

SUMMARY AND RECOMMENDATIONS

Technical assistance is a common strategy for encouraging andensuring the uptake of new knowledge and information andincreasingly it is becoming the vehicle for supporting organiza-tional and systems change. Stakeholders and consumers of TAalong with TA providers can be better prepared for the chal-lenge and for developing a functional partnership when thescope of the change initiative is clear, the resources match thescope of change, and when the TA strategies (e.g., Intensive,

Basic) match the desired outcomes. While the research eortsrelated to technical assistance impact are far from rigorous andconclusive, there are indicators that TA matters and there areemerging evaluation and research methods upon which futureinvestigations of TA processes and outcomes can be built.

As initiatives are planned itis helpful to rst assess thebreadth and depth of thechange needed to achievethe desired outcomes. ebreadth and depth of the

change should drive thetype of TA provided. Tohelp facilitate the discussionof factors, outcomes, and TArequirements, a TA AnalysisDiscussion Tool is providedbelow. is resource isprovided in order to generate a focused discussion amongthe implementation team members, the TA provider(s), andthe stakeholders regarding the key factors associated with theeort and degree of change required. e greater the change

required in relation to these factors the more likely it is thatIntensive TA will be required. While the process can be usedto generate a total average score, the score alone should not beused to make the decision. Rather it is the discussion among theparticipants and the agreement about the factors that will helpilluminate the depth and breadth of the change and guide thedecision about the intensity of TA required.

As initiatives areplanned it is helpul torst assess the breadthand depth o the change

needed to achieve thedesired outcomes. ebreadth and depth o

the change should drivethe type o TA provided.


14/17

12

CONSIDERATIONS RELATED TO THE NECESSARYINTENSITY TO ACCOMPLISH DESIRED OUTCOME:

DEGREE OF CHANGE REQ.

None Extensive

Instructions: For each item rate thedegree to which 1 2 3 4 5

New skill sets are required of practitioners to achieve outcomes for childrenand families

Practice change is required at multiple levels beyond practitioner level (e.g.,supervisor, administrator)

Changes in functional organizational supports are required (e.g., equipment,insurance, space, technology)

New thinking and acceptance of new ideas is required (e.g., inclusion, parentinvolvement, use of data), along with the letting go of current ways of work oradopting new values.

Changes in local infrastructure are required to support practice change andpractitioner competency (e.g., selection, training, coaching, availability anduse of data)

Policy changes, excluding funding, are required at multiple levels

Changes in funding are required to support the required competencyinfrastructure (e.g., selection, training, coaching, availability and use of data)

Changes in many parts of the system are required (e.g., new referral processes,new assessments, new partners, new services, new collaborations acrosssystems, closing services)

Changes in structures, policies, or funding eligibility are required at one ormore levels (e.g., program, district, state, Federal) to support direct service(e.g., practitioners work with children and families)

Broad systems change is required to achieve desired outcomes (e.g., statewidevs. pilot sites only)

Average Total Score (Closer to 1.0 indicates more Basic Technical Assistance required and closer to5.0 more Intensive Technical Assistance required)

Technical Assistance Analysis Discussion Tool

is resource is provided in order to generate a focused discussion among the implementing organization, the Technical Assis-tance provider(s), and the stakeholders regarding key considerations associated with the eort and degree of the change requirede greater the change required in relation to these items the more likely it is that Intensive Technical Assistance will be requiredWhile the process can be used to generate a total average score, the score should not be used to make the decision. Rather it ithe discussion among the participants that will help illuminate the depth and breadth of the change and will help to guide thedecision about the intensity of Technical Assistance required to achieve desired outcomes.


15/17

13

Americans with Disabilities Act of 1990, 42 U.S.C. 12101et seq. (West, 1993).

Barrett, S.B., Bradshaw, C.P., & Lewis-Palmer, T. (2008).

Maryland statewide PBIS initiative: systems,evaluation, and next steps.Journal o PositiveBehavior Interventions, 10; 105-114.

Barton, R. (Ed.). (2004, November). Technical assistanceguidelines. Northwest Regional EducationalLaboratory. Retrieved June 2, 2009, from http://www.nwrel.org/nwreport/2004-11/index.html

Choudhury, S. H. (2001). Fostering successful technicalassistance partnerships: Input paper for theUNCDF/SUM and UNDP Africa Global Meeting

May 30 to June 1, 2001Young and PromisingMicronance Institutions. New York: UnitedNations Capital Development Fund/Special Unitfor Micronance. Retrieved June 4, 2009, fromhttp://www.uncdf.org/english/micronance/uploads/thematic/02-Asa-SuccessfulTA-Final.pdf

Datnow, A., & Stringeld, S. (2000). Working togetherfor reliable school reform.Journal o Education orStudents Placed at Risk, 5(1), 183204.

Denton, C. A., Vaughn, S., & Fletcher, J. M. (2003).Bringing research-based practice in readingintervention to scale. Learning Disabilities Research &Practice, 18(3), 201-211.

Council of Chief State School Ocers (CCSSO). (2005).State systems o technical assistance delivery in specialeducation August 2005 questionnaire: Summary ondings. Washington, DC: Author. Retrieved June10, 2009, from www.k8accesscenter.org/documents/SETASurvey20062.doc

Dunlap, G., Strain, P.S., Fox, L., Carta, J.J., Conroy, M.,Smith, B., Kern, L., Hemmeter, J.L., Timm, M.A.,

McCart, A., Sailor, W., Markey, U., Markey, D.J.,Lardieri, S., & Sowell, C. (2006). Prevention andintervention with young childrens challengingbehavior: Perspectives regarding current knowledge.Behavioral Disorders, 32 (1), 29-45.

Durlak, J. A. (1998). Why program implementation isimportant.Journal o Prevention and Intervention inthe Community, 17(2), 5-18.

REFERENCES

Feinberg, M.E., Greenberg, M.T., Osgood, D.W., Anderson,A., & Babinski, L. (2002). e eects of trainingcommunity leaders in prevention science:Communities at Care in Pennsylvania.

Evaluationand Program Planning, 25, 245-259.

Feinberg, M. E., Ridenour, T. A., & Greenberg, M. T.(2008). e longitudinal eect of technical assistancedosage on the functioning of Communities atCare prevention boards in Pennsylvania. e Journalo Primary Prevention, 29(2), 145-165.

Fixsen, D. L., & Blase, K. A. (1993). Creating new realities:Program development and dissemination.Journal oApplied Behavior Analysis, 26(4), 597-615.

Fixsen, D. L., Blase, K. A., Horner, R., & Sugai, G. (2009,February). Intensive technical assistance. Scaling-upbrie #2. Chapel Hill, NC: e University of NorthCarolina at Chapel Hill, FPG Child DevelopmentInstitute, SISEP.

Fixsen, D. L., Blase, K. A., Timbers, G. D., & Wolf, M.M. (2001). In search of program implementation:792 replications of the Teaching-Family Model.In G. A. Bernfeld, D. P. Farrington & A. W.Leschied (Eds.), Ofender rehabilitation in practice:Implementing and evaluating efective programs(pp.

149-166). London: Wiley.

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M.& Wallace, F. (2005). Implementation research: Asynthesis o the literature. Tampa, FL: University ofSouth Florida, Louis de la Parte Florida MentalHealth Institute, e National ImplementationResearch Network (FMHI Publication #231).

Fox, L., Dunlap, G., Hemmeter, M.L., Joseph, G. &Strain, P. (2003). e Teaching Pyramid: A modelfor supporting social competence and preventingchallenging behavior in young children. YoungChildren, 58(4), 48-53.

Fullan, M. (1999). Change orces: e sequel. Philadelphia: Falmer

Fullan, M. (2007). e new meaning o educational change(4th ed.). New York: Teachers College Press.

Godfrey, M., Sophal, C., Kato, T., Piseth, L. V., Dorina, P.,Saravy, T., Savora, T., & Sovannarith, S. (2002).Technical assistance and capacity developmentin an aid-dependent economy: e experience ofCambodia. World Development, 30(3), 355-373.


16/17

14

Harbin, G. L., McWilliam, R. A., & Gallagher, J. J. (2000).Services for young children with disabilities andtheir families. In J.P. Shonko, & J.P. Meisels (Eds.),Handbook o early childhood intervention (2nd ed.).New York: Cambridge University Press.

Harchik, A. E., Sherman, J. A., Sheldon, J. B., & Strouse, M. C.(1992). Ongoing consultation as a method of improvingperformance of sta members in a group home.Journal

o Applied Behavior Analysis, 25, 599-610.

Hebbeler, K. M., & Wagner, M. (1998). e NationalIntervention Longitudinal Study (NEILS) designoverview. Retrieved April 30, 2008, from http://www.sri.com/neils/pdfs/neilsdgn.pdf

Hemmeter, M.L., & Fox, L. (2009). e Teaching Pyramid:A model for the implementation of classroompractices within a program-wide approach tobehavior support. NHSA Dialogue, 12(2), 133-147.

Hopkins, R. H., Mauss, A. L., Kearney, K. A., & Weisheit,R. A. (1988). Comprehensive evaluation of a modelalcohol education curriculum. Journal of Studies onAlcohol, 49, 3850.

Hurth, J., Shaw, E., Izeman, S.G., Whaley, K. & Rogers(1999). Areas of agreement about eective practicesamong programs serving young children with autismspectrum disorders. Inants and Young Children,12(20): 17-26.

Individuals with Disabilities Education Act Amendments of1997, 20 U.S.C. 1400 et seq.

International Monetary Fund. (2002). Capacity building:Lessons or Arica. Washington, DC: Arthur.Retrieved June 11, 2009, from http://www.imf.org/external/np/tr/2002/tr020912a.htm

Joyce, B., & Showers, B. (2002). Student achievement throughstaf development(3rd ed.). Alexandria, VA: Associationfor Supervision and Curriculum Development.

Kahn, L., Hurth, J., Kasprzak, C. M., Diefendorf, M.J., Goode, S. E., & Ringwalt, S. S. (2009). eNational Early Childhood Technical AssistanceCenter model for long-term systems change. Topicsin Early Childhood Special Education, 29(1), 24-39.

Kavanagh, D. J., Spence, S. H., Strong, J., Wilson, J., Sturk,H., & Crow, N. (2003). Supervision practices inallied mental health: Relationships of supervisioncharacteristics to perceived impact and job satisfaction.Mental Health Services Research, 5(4), 187-195.

Kegler, M.C., Steckler, A., Malek, S.H., & McLeroy, K.(1998). A multiple case study of implementationin 10 local Project ASSIST coalitions in NorthCarolina. Health Education Research, 13, 225-238.

Kelly, J. A., Somlai, A. M., DiFranceisco, W. J., Otto-Salaj,L. L., McAulie, T. L., Hackl, K. L., et al. (2000).Bridging the gap between the science and serviceof HIV prevention: Transferring eective research-

based HIV prevention interventions to communityAIDS service providers.American Journal o PublicHealth, 90(7), 1082-1088.

Laforge, R.G., Velicer, W.F., Richmond, R.L., &Owen, N.(1999). Stage distributions for ve health behaviors inthe USA and Australia. Preventive Medicine, 28, 61-74

Marks, S. U., & Gersten, R. (1998). Engagement anddisengagement between special and general educatorsAn application of Miles and Hubermans cross-caseanalysis. Learning Disability Quarterly, 21(1), 34-56.

McCormick, K. M., & Brennan, S. (2001). Mentoring thenew professional in interdisciplinary early childhoodeducation: e Kentucky Teacher InternshipProgram. Topics in Early Childhood SpecialEducation, 21(3), 131-144.

Mitchell, R., Florin, P., & Stevenson, J. (2002). Supportingcommunity-based prevention and health promotioninitiatives. Developing eective technical assistancesystems. Health, Education, and Behavior, 29, 620-639.

Mitchel, M. E., Hu, T.-W., McDonnell, N. S., & Swisher,

J. D. (1984). Cost-eectiveness analysis of aneducational drug abuse prevention program.Journalo Drug Education, 14, 271291.

Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D.(2003). Fidelity criteria: Development, measurement,and validation.American Journal o Evaluation,24(3), 315-340.

National Dissemination Center for Children withDisabilities ( 2009). Disseminating or Impact, 1 (2),1-4. Retrieved May 14, 2009, from http://www.

nichcy.org/Documents/dissem-cop-docs/Issue2-StartingDialogue_nal_web.pdf

Odom, S.L. (2000). Preschool inclusion: what we know andwhere we go from here. Topics in Early ChildhoodSpecial Education, (20)1, 20-27.

ODonnell, C. L. (2009). Dening, conceptualizing,and measuring delity of implementation andits relationship to outcomes in K-12 curriculumintervention research. Review o Educational Research78(1), 33-84.


17/17

Paul, C.L., & Redman, S. (1997). A review of print materialin changing health-related knowledge, attitudes, andbehavior. Health Promotion Journal o Australia, 7(2),91-99.

Prochaska, J.O., & DiClemente, C.C. (1983). Stages andprocesses of self-change of smoking: Toward anintegrative model of change.Journal o Consultingand Clinical Psychology, 51,390-395.

Prochaska, J.M., Prochaska, J.O., & Levesque, D. (2001). Atranstheoretical approach to changing organizations.Administration and Policy in Mental Health, 28 (4),March, 247-261.

Ringwalt, S. (2008). Developmental screening and assessmentinstruments with an emphasis on social and emotionaldevelopment or young children ages birth throughve. Chapel Hill: e University of North Carolina,FPG Child Development Institute, National EarlyChildhood Technical Assistance Center.

Schoenwald, S. K., Sheidow, A. J., & Letourneau, E. J. (2004).Toward eective quality assurance in evidence-basedpractice: Links between expert consultation, therapistdelity, and child outcomes.Journal o Clinical Childand Adolescent Psychology, 33(1), 94-104.

Tortu, S., & Botvin, G. J. (1989). School-based smokingprevention: e teacher training process. PreventionMedicine, 18, 280-289.

Trohanis, P. L. (2004). Ideas about the change process: Ashort synthesis o thinking points gleaned rom the

literature, practice and over 35 years o experience.Unpublished manuscript.

Velicer, W. F., DiClemente, C. C., Prochaska, J. O., &Brandenberg, N. (1985). A decisional balance measurefor assessing and predicting smoking status.Journal oPersonality and Social Psychology, 48, 1279-1289.

Velicer, W.F., Fava, J.L., Prochaska, J.O., Abrams, D.B.,Emmons, K.M., & Pierce, J.P. (1995). Distributionof smokers by stage in three representative samples.Preventive Medicine, 24, 401-411.

Winton, P. & Catlett, C. (2009). Statewide eorts to enhanceearly childhood personnel preparation programsto support inclusion, overview and lessons learned.Inants and Young Children (22) 1, 63-70.

Yin, R.K., Gwaltney, M.K., Hare, M., & Butler, M.O.(1999). Final Report: Evaluation o the CDC-supportedtechnical assistance network or community planning.Bethesda, MD: COSMOS Corporation.