beyond coaching : a tiered model for technical assistance...
TRANSCRIPT
Beyond Coaching: A T iered M odel for T echnical A s s is tance in a QR IS
NAEYC National Institute for Early Childhood Professional Development
June 7th , 2015
Contact Information
• Sybille Guy
• [email protected] • Tom Udell
• [email protected] • Robyn Lopez Melton
Introduction
• QRIS Administrators • Program Staff • Accreditation Staff • Head Start Staff • Researchers/Evaluators • Coaches
Outcome and Objectives
Understand a tiered model of technical assistance used to provide supports in a Quality Rating and Improvement System (QRIS).
• Examine the roles, responsibilities and qualifications of Technical Assistance Providers, Quality Improvement Specialists(QIS).
• Discuss a technical assistance model tiered across universal, targeted and intensive levels of support used in a Quality Rating and Improvement System (QRIS).
• Discuss initial findings from the process evaluation on the effectiveness of the model.
Oregon’s QRIS Overview
Oregon’s QRIS 5 Tier Bui lding Block S ys tem
Licensed
Ince
ntiv
es fo
r qu
ality
ratin
g
Commitment to Quality Supp
orts
for Q
ualit
y Im
prov
emen
ts
QRIS Components
Building Blocks with 5 Tiers
5 Domains
2 Versions
of Materials
Quality Improvement Quality
Improvement Plans
Self- Assessments
Supports
Quality Rating Portfolio
System of Documentation
Incentives
QRIS Domains and Standards
• 12 Standards Children’s Learning and Development
• 6 Standards Health and Safety
• 4 Standards Family Partnerships
• 5 Standards Personnel Qualifications
• 6 Standards Administration and Business Practices
Portfol io System
• Balance of evidence • Data • Documentation • Report • Observation
• Reviewed by experts
Basics of the QRIS Process
Commit
• QRIS Increasing Quality Training
• Application • Self Assessment
Participate
• Quality Improvement Plan
• Quality Improvement Supports
• Technical Assistance with Portfolio
• Financial Supports of $1000-$2000
Achieve
• Submit Portfolio to Western Oregon University
• Portfolio reviewed by experts
• Receive a rating • Financial
Incentives between $500-$2500
Oregon’s QRIS Field Test
•Conclusion of the first validation study. Beginning of second validation study.
Early 2015
•Statewide rollout of field test
March 2014
•Teen Parent Programs •Crosswalks completed,
launched streamlined process for accredited programs
•Head Start (late 2013)
Fall 2013
•Pilot began in 4 regions
January 2013
Participation Stats M ay 31, 2015
Participation Stats
Role of the Qual i ty Improvement S pecialis t
TA on quality improvements TA on portfolio
development and rating
Programs and professional
development Coaching
Quality Improvement Specialist
Quali fication of the Qual i ty Im provem ent S pecialis t
• 1 year of experience supporting adult professional development
• Preferred Qualifications: Oregon Registry Master Trainer and experience working with diverse populations
• 3 years of experience in the field of Childhood Care and Education
• Associates Degree in ECE or related field/Oregon Registry Step 9
• Preferred Qualifications: Bachelor Degree in ECE or related field/Oregon Registry Step 10 or higher
Tiered Technical A s s is tance
Review of a technical assistance model tiered across universal, targeted and intensive levels of support
Why Tiered Technical Assistance?
• Used with the Office of Special Education Programs TA & Projects (NCDB).
• Experience with applying to a state project supporting inclusion.
• Model with promise of being cost effective.
• Need to orient the field away from focusing on individual on-site TA.
Overview of TA Model
Universal
Targeted
Intensive
Universal
• Cost effective for meeting broad needs in the field
• Informational training sessions offered to groups with content that is general and not necessarily individualized or customized for specific groups
Targeted
• Cost effective strategies for providing needs based assistance
• Support is provided based on needs common to multiple recipients and delivered in group trainings and/or work sessions
• Support is based on identifying individual programs needs, analyzing needs across multiple programs and grouping programs receiving support
Intensive
• Individual on-going coaching and consultation that includes on-site visits to programs
• This support is based on individual needs that are
unique to the program or are best addressed with an individual program
Access
Universal
Targeted
Intensive
• Individual or program initiated
• QIS initiated
Audience
Universal
Targeted
Intensive
• Broad audience of independent users accessing through their own initiative.
• Multiple recipients with an identified common need accessing through invitation and planning.
• Individual or program with established relationship with TA provider and an identified specific need.
Outcome
Universal
Targeted
Intensive
• Awareness and knowledge outcomes
• Knowledge and skill outcomes
• Skill outcomes
QRIS Examples
Universal
Targeted
Intensive
• Increasing Quality Training • Web-based TA provided by TRI
• Work Sessions • Trainings
• Individualized Support • On-site Visits
Is it universal, targeted or intensive technical assistance?
Activi ty
Tiering can be a continuum
Universal
Workshop on family involvement
Targeted
Training on developing family involvement policies
Intensive
Assisting a program to set up a parent input meeting
Family Involvement
Tiering can be a continuum
Universal
Workshop on Adult-Child Interactions
Targeted
Training for Directors on coaching for adult child interactions
Intensive
Observing and providing feedback in a program.
Adult-Child Interactions
Principles for using Tiered TA
• Based on knowing the needs of programs
• Informed by trends
• Guided by program’s Quality Improvement Plan
• Influenced by the type of change/improvement
Review QIP
Principles for using Tiered TA
• Based on knowing the needs of programs
• Informed by trends
• Guided by program’s Quality Improvement Plan
• Influenced by the type of change/improvement
Activi ty What level of TA is appropriate?
Ini t ial Findings Process evaluations data to date
Signi ficantly More Contacts by QISs with Program s who A chieved R atings
4.88
2.14 2.22
12.63
6.03 4.38
0
2
4
6
8
10
12
14
16
CC CF RF
Average # of Contacts
Not Achieved Rating Achieved Rating
CC = Certified Center; CF = Certified Family; RF = Registered Family
Signi ficantly More Time Spent by QISs with Program s who A chieved R atings
148.06 99.14
127.26
494.5
310.06 269.62
0
100
200
300
400
500
600
CC CF RF
Overall Average (Time Spent in Minutes)
Not Achieved Rating Achieved Rating
CC = Certified Center; CF = Certified Family; RF = Registered Family
Dai ly Contact: Phone
[CATEGORY NAME]/
Intensive [PERCENTA
GE]
[CATEGORY NAME]
[PERCENTAGE]
Dai ly Contact: Emai l
Targeted/Intensive 49%
Universal 51%
Dai ly Contact: In Person
Targeted/Intensive 72%
Universal 28%
TA: An Analysis
0.00
50.00
100.00
150.00
200.00
250.00
Intensive Targeted Universal
• Time in minutes
Type N
Intensive 798
Targeted 143
Universal 137
*from logs from September 2013-February 2014 *subsection of all the TA for analysis
Effort: T otal T im e s pent (in m inutes )/# of Program s
Type Average Time To Prepare
Average Training
Time
Average Effort (Total
Time/# of Programs)
Intensive 40.49 98.66 137.73 Targeted 137.16 146.64 60.72 Universal 32.67 69.42 39.01
Df SSE MSE F P-value
Type 2 1624607 812304 67.18 <2e-16
Residuals 1075 12997970 12091
Total n=2078 contacts 0.00
20.00
40.00
60.00
80.00
100.00
120.00
140.00
160.00
Intensive Targeted Universal
Case Study: Curriculum Cohort
• Targeted TA • Curriculum Cohort • 11 Programs (2 Small Family, 5 Medium Family, 4 Centers) • 21 Hours of Training
• QIS recognized a need for training in Creative Curriculum
• 18 hours spent creating training
Case Study: Curriculum Cohort
Universal
Targeted
Intensive
• Increasing Quality Training • Café • E-mail • 21 hours of training on the
Creative Curriculum • 90 minutes of on-site
support
• 11 Programs
• 2 Programs
Case Study: Curriculum Cohort
• 10 Submitted Portfolios
• 10 received a Star Rating
• Four (2 Centers, 1 Medium and 1 Small family) joined a new cohort to achieve a higher star
Case Study: Curriculum Cohort
Universal
Targeted
Intensive
• 40 minutes (estimated bases on
average times) • 213 minutes (18 hours spent creating
training and 21 hours delivering)
• 8 minutes
• 261 per program total
What we have learned
• More time spent with programs who achieved ratings. • Intensive and Universal TA are similar in time needed
to prepare. Targeted TA takes more time to prepare and deliver.
• Targeted involves more programs receiving TA at the
same time = more cost effective than Intensive.
Next Steps
What we need to ask: • Is the TYPE of TA influencing the number of ratings? What we need to do: • Analyze the Type of TA and its influence on number of
ratings (General Linear Model analysis).
Questions?
Thank You!