beyond coaching : a tiered model for technical assistance...

Beyond Coaching: A T iered M odel for T echnical A s s is tance in a QR IS

NAEYC National Institute for Early Childhood Professional Development

June 7th , 2015

Contact Information

• Sybille Guy

• guys@wou.edu • Tom Udell

• udellt@wou.edu • Robyn Lopez Melton

• meltonr@wou.edu

Introduction

• QRIS Administrators • Program Staff • Accreditation Staff • Head Start Staff • Researchers/Evaluators • Coaches

Outcome and Objectives

Understand a tiered model of technical assistance used to provide supports in a Quality Rating and Improvement System (QRIS).

• Examine the roles, responsibilities and qualifications of Technical Assistance Providers, Quality Improvement Specialists(QIS).

• Discuss a technical assistance model tiered across universal, targeted and intensive levels of support used in a Quality Rating and Improvement System (QRIS).

• Discuss initial findings from the process evaluation on the effectiveness of the model.

Oregon’s QRIS Overview

Oregon’s QRIS 5 Tier Bui lding Block S ys tem

Licensed

Ince

ntiv

es fo

r qu

ality

ratin

g

Commitment to Quality Supp

orts

for Q

ualit

y Im

prov

emen

ts

QRIS Components

Building Blocks with 5 Tiers

5 Domains

2 Versions

of Materials

Quality Improvement Quality

Improvement Plans

Self- Assessments

Supports

Quality Rating Portfolio

System of Documentation

Incentives

QRIS Domains and Standards

• 12 Standards Children’s Learning and Development

• 6 Standards Health and Safety

• 4 Standards Family Partnerships

• 5 Standards Personnel Qualifications

• 6 Standards Administration and Business Practices

Portfol io System

• Balance of evidence • Data • Documentation • Report • Observation

• Reviewed by experts

Basics of the QRIS Process

Commit

• QRIS Increasing Quality Training

• Application • Self Assessment

Participate

• Quality Improvement Plan

• Quality Improvement Supports

• Technical Assistance with Portfolio

• Financial Supports of $1000-$2000

Achieve

• Submit Portfolio to Western Oregon University

• Portfolio reviewed by experts

• Receive a rating • Financial

Incentives between $500-$2500

Oregon’s QRIS Field Test

•Conclusion of the first validation study. Beginning of second validation study.

Early 2015

•Statewide rollout of field test

March 2014

•Teen Parent Programs •Crosswalks completed,

launched streamlined process for accredited programs

•Head Start (late 2013)

Fall 2013

•Pilot began in 4 regions

January 2013

Participation Stats M ay 31, 2015

Participation Stats

Role of the Qual i ty Improvement S pecialis t

TA on quality improvements TA on portfolio

development and rating

Programs and professional

development Coaching

Quality Improvement Specialist

Quali fication of the Qual i ty Im provem ent S pecialis t

• 1 year of experience supporting adult professional development

• Preferred Qualifications: Oregon Registry Master Trainer and experience working with diverse populations

• 3 years of experience in the field of Childhood Care and Education

• Associates Degree in ECE or related field/Oregon Registry Step 9

• Preferred Qualifications: Bachelor Degree in ECE or related field/Oregon Registry Step 10 or higher

Tiered Technical A s s is tance

Review of a technical assistance model tiered across universal, targeted and intensive levels of support

Why Tiered Technical Assistance?

• Used with the Office of Special Education Programs TA & Projects (NCDB).

• Experience with applying to a state project supporting inclusion.

• Model with promise of being cost effective.

• Need to orient the field away from focusing on individual on-site TA.

Overview of TA Model

Universal

Targeted

Intensive

Universal

• Cost effective for meeting broad needs in the field

• Informational training sessions offered to groups with content that is general and not necessarily individualized or customized for specific groups

Targeted

• Cost effective strategies for providing needs based assistance

• Support is provided based on needs common to multiple recipients and delivered in group trainings and/or work sessions

• Support is based on identifying individual programs needs, analyzing needs across multiple programs and grouping programs receiving support

Intensive

• Individual on-going coaching and consultation that includes on-site visits to programs

• This support is based on individual needs that are

unique to the program or are best addressed with an individual program

Access

Universal

Targeted

Intensive

• Individual or program initiated

• QIS initiated

Audience

Universal

Targeted

Intensive

• Broad audience of independent users accessing through their own initiative.

• Multiple recipients with an identified common need accessing through invitation and planning.

• Individual or program with established relationship with TA provider and an identified specific need.

Outcome

Universal

Targeted

Intensive

• Awareness and knowledge outcomes

• Knowledge and skill outcomes

• Skill outcomes

QRIS Examples

Universal

Targeted

Intensive

• Increasing Quality Training • Web-based TA provided by TRI

• Work Sessions • Trainings

• Individualized Support • On-site Visits

Is it universal, targeted or intensive technical assistance?

Activi ty

Tiering can be a continuum

Universal

Workshop on family involvement

Targeted

Training on developing family involvement policies

Intensive

Assisting a program to set up a parent input meeting

Family Involvement

Tiering can be a continuum

Universal

Workshop on Adult-Child Interactions

Targeted

Training for Directors on coaching for adult child interactions

Intensive

Observing and providing feedback in a program.

Adult-Child Interactions

Principles for using Tiered TA

• Based on knowing the needs of programs

• Informed by trends

• Guided by program’s Quality Improvement Plan

• Influenced by the type of change/improvement

Review QIP

Principles for using Tiered TA

• Based on knowing the needs of programs

• Informed by trends

• Guided by program’s Quality Improvement Plan

• Influenced by the type of change/improvement

Activi ty What level of TA is appropriate?

Ini t ial Findings Process evaluations data to date

Signi ficantly More Contacts by QISs with Program s who A chieved R atings

4.88

2.14 2.22

12.63

6.03 4.38

0

2

4

6

8

10

12

14

16

CC CF RF

Average # of Contacts

Not Achieved Rating Achieved Rating

CC = Certified Center; CF = Certified Family; RF = Registered Family

Signi ficantly More Time Spent by QISs with Program s who A chieved R atings

148.06 99.14

127.26

494.5

310.06 269.62

0

100

200

300

400

500

600

CC CF RF

Overall Average (Time Spent in Minutes)

Not Achieved Rating Achieved Rating

CC = Certified Center; CF = Certified Family; RF = Registered Family

Dai ly Contact: Phone

[CATEGORY NAME]/

Intensive [PERCENTA

GE]

[CATEGORY NAME]

[PERCENTAGE]

Dai ly Contact: Emai l

Targeted/Intensive 49%

Universal 51%

Dai ly Contact: In Person

Targeted/Intensive 72%

Universal 28%

TA: An Analysis

0.00

50.00

100.00

150.00

200.00

250.00

Intensive Targeted Universal

• Time in minutes

Type N

Intensive 798

Targeted 143

Universal 137

*from logs from September 2013-February 2014 *subsection of all the TA for analysis

Effort: T otal T im e s pent (in m inutes )/# of Program s

Type Average Time To Prepare

Average Training

Time

Average Effort (Total

Time/# of Programs)

Intensive 40.49 98.66 137.73 Targeted 137.16 146.64 60.72 Universal 32.67 69.42 39.01

Df SSE MSE F P-value

Type 2 1624607 812304 67.18 <2e-16

Residuals 1075 12997970 12091

Total n=2078 contacts 0.00

20.00

40.00

60.00

80.00

100.00

120.00

140.00

160.00

Intensive Targeted Universal

Case Study: Curriculum Cohort

• Targeted TA • Curriculum Cohort • 11 Programs (2 Small Family, 5 Medium Family, 4 Centers) • 21 Hours of Training

• QIS recognized a need for training in Creative Curriculum

• 18 hours spent creating training

Case Study: Curriculum Cohort

Universal

Targeted

Intensive

• Increasing Quality Training • Café • E-mail • 21 hours of training on the

Creative Curriculum • 90 minutes of on-site

support

• 11 Programs

• 2 Programs

Case Study: Curriculum Cohort

• 10 Submitted Portfolios

• 10 received a Star Rating

• Four (2 Centers, 1 Medium and 1 Small family) joined a new cohort to achieve a higher star

Case Study: Curriculum Cohort

Universal

Targeted

Intensive

• 40 minutes (estimated bases on

average times) • 213 minutes (18 hours spent creating

training and 21 hours delivering)

• 8 minutes

• 261 per program total

What we have learned

• More time spent with programs who achieved ratings. • Intensive and Universal TA are similar in time needed

to prepare. Targeted TA takes more time to prepare and deliver.

• Targeted involves more programs receiving TA at the

same time = more cost effective than Intensive.

Next Steps

What we need to ask: • Is the TYPE of TA influencing the number of ratings? What we need to do: • Analyze the Type of TA and its influence on number of

ratings (General Linear Model analysis).

Questions?

Thank You!