commonwealth of kentucky community early childhood … evaluation report - 2017.pdftable 2 provides...

46
Kentucky CECC Evaluation 1 Commonwealth of Kentucky Community Early Childhood Council Evaluation Study July 2017 Overview Ghazvini Consulting Services responded to a solicitation from the Commonwealth of Kentucky and was awarded a contract to complete a Community Early Childhood Council Evaluation Study to commence on July 1 and to be completed by July 31, 2017. The approved evaluation design included conducting two on-line surveys, one for CECC members and one for CECC Request for Application reviewers; interviews with CECC leadership; a review of state-local partnerships in other states; a review of standards and system components in Kentucky; an analysis of findings; development of recommendations based on the findings; and a revised draft CECC Request for Application based on the recommendations. The following report documents the work as required. A revised draft CECC Request for Application is provided separately.

Upload: others

Post on 18-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 1

Commonwealth of Kentucky Community Early Childhood Council Evaluation Study

July 2017

Overview

Ghazvini Consulting Services responded to a solicitation from the

Commonwealth of Kentucky and was awarded a contract to complete a Community Early Childhood Council Evaluation Study to commence on July 1 and to be completed by July 31, 2017. The approved evaluation design included conducting two on-line surveys, one for CECC members and one for CECC Request for Application reviewers; interviews with CECC leadership; a review of state-local partnerships in other states; a review of standards and system components in Kentucky; an analysis of findings; development of recommendations based on the findings; and a revised draft CECC Request for Application based on the recommendations. The following report documents the work as required. A revised draft CECC Request for Application is provided separately.

Page 2: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 2

Analysis of Community Early Childhood Council Surveys

Introduction

Community Early Childhood Councils (CECCs) were established in

Kentucky in KIDS NOW legislation in 2000. During the 2016-17 fiscal year, 75

councils serving 110 of Kentucky’s 120 counties were awarded $1,274,113.64 (one

council returned their funding after the awards were released). The Kentucky

Governor’s Office of Early Childhood works in partnership with the councils to

mobilize local community members to build innovative, collaborative

partnerships that promote school readiness for children and families. CECCs

bring local perspective and assets to Kentucky’s early childhood system

development and implementation. Council members work together to identify

local needs, develop strategies to address those needs that maximize state and

local resources, and measure their progress.

CECCs apply annually for funding through a written Request for

Application process. The focus of the grants has changed moderately over the

years, and this evaluation of the funding system is the first one that has been

completed. The Kentucky Governor’s Office of Early Childhood completed a

formal procurement process to identify a third-party reviewer to evaluate the

method and process of local council funding and make recommendations to

better align the state-local partnership with Kentucky’s goal of school readiness.

Method

A survey was developed to solicit input from local CECC members

regarding the state required processes and requirements relative to local council

operations. Survey questions addressed the Request for Application, local

council operations, reporting requirements, and perceived local impact. Experts

in research design recommend using multiple methods of feedback to increase

Page 3: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 3

the effectiveness of evaluation design.1 As a result, this evaluation utilized

interviews in addition to on-line surveys. More in-depth input was sought

through interviews with CECC leaders. At least two leaders (typically the chair,

co-chair, past-chair, Secretary, and/or Treasurer) from a sample of 29 CECCs

were invited to participate in an interview. A separate interview protocol was

developed for Request for Application reviewers to garner their input regarding

the application processes. Copies of the survey and interview questions are

included in Appendix A. The following is the findings relative to each of these

evaluation efforts.

CECC Member Survey

Respondents

Email invitations with a link to complete an on-line survey were

successfully sent to 1,192 individuals who served as a CECC member during

fiscal years 2015-16 and/or 2016-172. Two hundred and sixty-one individuals

responded to the survey, a response rate of 22%. Online survey response rates

vary from 10% to 30%, on average, and this response allows a 95% confidence

level with a 5.5 percent margin of error. Determining the response rates from

single versus multi-county councils is difficult, as approximately 25% of

respondents did not respond to the question asking for the name of their CECC.

However, approximately 40% of the respondents replied ‘not applicable’ to a

question that specifically addressed multi-county councils. Therefore, it appears

that there was a good balance of responses between CECC members representing

single-county and multi-county councils.

1 Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? In

Assessment and Evaluation in Higher Education, 33 (3), 301-317. 2 Due to an incomplete electronic list, not all 2015 CECC Members were sent a survey.

Page 4: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 4

Results

Application Processes

Most respondents replied favorable to questions regarding the Request for

Applications. Approximately 82% agreed or strongly agreed that the RFA

process is fair and equitable with only 5% responding disagree or strongly

disagree (approximately 13% neither agreed nor disagreed). Similar responses

were given when asked if the funding process was equitable for both new and

returning CECCs.

Figure 1. CECC Member Perceptions of Review Process Transparency.

There were slight differences in responses when asked about the

transparency of the review process with approximately 69% agreeing or strongly

agreeing to the transparency of the process and approximately 10% disagreeing

or strongly disagreeing (see Figure 1). Similar response rates were found to the

question addressing the clarity of the review scoring parameters (65% were in

agreement that it was clear and understandable, and 11% disagreed). When

asked about the fairness of the review process, 74% agreed or strongly agreed

that it was fair, and 5% disagreed or strongly disagreed.

Other questions addressed the clarity of wording in the RFA and the

timeline. A summary of these responses is included in Table 1.

Page 5: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 5

Table 1. CECC Member Survey Responses regarding the Request for Applications.

Question Strongly Agree

Agree Neither Agree Nor Disagree

Disagree Strongly Disagree

The goals and requirements of the RFA are understandable

14% 62% 15.5% 7.5% 1%

Terminology in the RFA is defined and clearly understood

14% 56% 20% 8% 2%

The RFA timeline for response is appropriate

13% 52% 17% 14% 4%

State Supports to Community Early Childhood Councils

Once again, the majority of responses were favorable regarding state-level

supports provided to local councils. When asked about the frequency of

communication between the Kentucky Governor’s Office of Early Childhood and

local councils, 72% were satisfied and only 12% were dissatisfied (see Figure 2).

The responses were very similar to a question regarding satisfaction with the

communication methods and mechanisms.

Figure 2. CECC Member Perceptions of Communication Frequency.

Page 6: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 6

Several questions were asked about training and technical assistance

supports provided to local councils. There was a slight increase in

dissatisfaction. There were significant differences in responses between single-

county and multi-county councils regarding the two questions relative to the

provision of technical assistance to councils. In terms of both the sufficiency and

quality of technical assistance opportunities, there was significantly (p < .05)

greater agreement to these statements by multi-county respondents. Table 2

provides an overview of these responses.

Table 2. CECC Survey Responses regarding Training and Technical Assistance.

Question Strongly Agree

Agree Neither Agree Nor Disagree

Disagree Strongly Disagree

Training provided to CECCs is sufficient

12.5% 46% 24% 14.5% 3%

Training provided to CECCs is of high quality

15% 44% 33% 6% 2%

Technical assistance provided to CECCs is sufficient

11% 49% 31% 7% 2%

Technical assistance provided to CECCs is of high quality

9% 47% 36.5% 6.5% 1%

CECC members were asked about School Readiness Summit participation.

The School Readiness Summits seek to give Superintendents and/or Assistant

Superintendents the opportunity to lead a collaborative team of representatives

from child care, Preschool, Head Start, CECC, and other district and community

partners to create 30, 60, and 90 day Action Plans to increase school readiness in

their communities. For the 2016 School Readiness Summit and the Mid-Year

School Readiness Summit, Quality Improvement Grant funding was available to

Page 7: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 7

assist with plan implementation. Since 2015, Kentucky has hosted two School

Readiness Summits as well as a Mid-Year School Readiness Summit with a total

of 46 districts represented.

When CECC members were asked whether the School Readiness Summit

provided helpful support to the work of CECCs, 46% agreed or strongly agreed

with the statement, 7.5% disagreed or strongly disagreed, 22% neither agreed nor

disagreed, and 24.5% responded that the question was not applicable. There was

also a significant difference in responses on this question between single and

multi-county councils (p < .05) with single-county councils less likely to agree

(34%) than multi-county councils (53.5%), perhaps due to multi-county councils

having more potential invitations to participate in a Summit than their single-

county colleagues.

Funding and Reporting Requirements

Council members also answered favorably to questions regarding funding

and reporting requirements. Most respondents agreed that the CECC funding

formula was clear (61% agreed or strongly agreed, 10% disagreed or strongly

disagreed, and 29% seemed unsure, indicating neither agree nor disagree) and

that funding requirements and processes were clear (69% were in agreement,

8.5% were in disagreement and 22.5% neither agreed nor disagreed). The budget

amendment process is also clear according to the majority of respondents (69.5%

in agreement and 7% in disagreement).

Overall reporting requirements are clear according to the majority of

respondents with 75% agreeing or strongly agreeing and 6.5% disagreeing or

strongly disagreeing. There was also strong agreement that reporting

requirements effectively communicate the progress and impact of local council

work (69.5% agreeing, 6% disagreeing, and 24.5% neither agreeing nor

disagreeing). Mid-year correction and improvement processes were identified as

Page 8: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 8

helpful, with 65% in agreement and 4.5% in disagreement (over 30% appeared

unsure).

When asked about the appropriateness of data collection requirements,

78% of respondents were in agreement that such requirements were appropriate,

with only 6% disagreeing. Slightly more favorable responses were found when

respondents were asked if the data collected was useful to Council work (82% in

agreement and only 3% in disagreement).

Perceptions regarding Council Work

Respondents replied very favorably to questions regarding the local value

of their work. Members strongly agreed with the statement that the CECC work

is valuable to the community (64% indicated strongly agree, 31% responded

agree, less than 1% indicated disagree, and less than 1% responded strongly

disagree). When asked if their CECC provides high quality and effective work,

49.5% strongly agreed and 42% agreed. Only 3% disagreed with this statement.

Responses were very similar to a statement about the community recognizing the

value of school readiness (89% in agreement and 5% in disagreement).

One question asked if the CECC effectively used volunteers in the

community, and again, responses were very favorable (79.5% agreed and 8.5%

disagreed). Another question was directed specifically to multi-county councils

and asked whether each county was sufficiently represented and addressed in

council planning and implementation of goals. Thirty-nine percent of

respondents indicated that the question was not applicable, and thus are likely

from single-county coalitions. Removing those respondents from the response

yields a favorable response rate with 66% agreeing or strongly agreeing with the

statement and 11% disagreeing or strongly disagreeing with the statement (23%

responded neither agree nor disagree).

Page 9: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 9

Open-ended Questions

Three open-ended questions were asked regarding the strengths,

challenges, and system recommendations. A summary of responses to these

questions is provided below.

What do you see as the major strengths of the CECC system?

Most responses spoke to the roles of CECCs in bringing the community

partners together and in advancing school readiness awareness. Increased

community collaboration with a multitude of different partners was noted often

as well as outreach and support to families with young children. Respondents

noted that the council helps partners prioritize, reduce duplication, and better

share and leverage resources. Several respondents noted the importance of local

planning and decision-making in order to meet the diverse local needs across

Kentucky. The importance of supporting early literacy, outreach to hard-to-

reach families, increased community involvement and awareness, and additional

funding to support the development of young children and their families was

noted.

What do you see as the major challenges of the CECC system?

Common responses to this question addressed the difficulty of getting

sufficient numbers of council members involved and assisting with council

activities and the need for additional funding. Many respondents noted that the

majority of Council work is completed by a relatively small number of members.

The volunteer nature of the work was noted as a contributing factor to the

difficulty of getting council work completed. Several respondents reported that

the requirements were too demanding and there is too much paperwork for a

volunteer board. One respondent suggested that funding is needed for paid staff

to ensure proper tracking of data collection and outcomes. Concerns were also

raised about the difficulty of getting child care providers and some other

Page 10: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 10

partners (e.g., business and government representatives) to participate on the

councils.

In addition to the challenge of limited funding, several respondents

addressed challenges relative to changes and delays in funding timelines, lack of

communication regarding funding issues, the uncertainty of funding, gaps in

funding, and the negative impact of funding delays on local efforts. Many of the

responses noted challenges relative to data collection. Several respondents noted

the recent change in priorities that removed the councils’ ability to utilize

funding to support professional development for child care providers as a

concern.

Difficulties in reaching some families and in getting families involved in

council activities were raised. Special challenges relative to outreach and

transportation were noted for rural areas. Another challenge that was reported

related to the difficulty of making an impact given the diversity of families and

communities. The special challenges of multi-county councils were noted given

the diversity of needs. One respondent also noted the inability to exchange

information about specific families due to confidentiality issues.

Concerns regarding the fairness of the Request for Application review

process were raised, with one respondent questioning the adequacy of reviewer

training and oversight. Several respondents raised concerns regarding the

infrequency of communication, training and assistance from the Kentucky

Governor’s Office of Early Childhood as well as the inconsistency in

expectations. Turnover of state level staff was noted as a challenge by a few

respondents. A few respondents also noted the lack of training and support for

local chairs, particularly new ones.

What are your recommendations of improvement to the CECC system?

Respondents recommended a broader array of supports and services,

including more programs for infants, for school-age children during out-of-

Page 11: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 11

school time, for the provision of professional development training and supports,

and provision of Strengthening Families Framework. A statewide media

campaign regarding early brain development was recommended as well as

continued focus and supports for hard-to-reach children. There was also a

recommendation to focus funding on activities that are intensive and frequent

enough in order to improve impact.

Improvements to the Request for Application and funding processes were

recommended, specifically addressing consistent timelines and expectations as

well as improved reviewer training and protocols and sharing of scoring sheets.

One respondent recommended aligning the funding and services of the CECC

RFA with the Family Resource Center and the Preschool Partnership Grants as

many of the same community members are involved in all three efforts. A two-

year funding cycle was also recommended. Additional supports for small, rural

counties where grant writers are not readily available to help were also

recommended. More local discretion and fewer restrictions on funding were

noted.

There were a number of recommendations for improving the relationships

between CECCs and the Kentucky Governor’s Office of Early Childhood. More

frequent communications and supports from the Kentucky Governor’s Office of

Early Childhood were requested, including the provision of site visits.

Communications regarding staff and policy changes and updates were

recommended as well.

Recommendations regarding the provision of training and assistance to

new chairs, training regarding council operations (e.g., running a meeting,

engaging members, strategic planning), and assistance finding additional

funding opportunities were made. Special assistance to struggling councils was

recommended. Opportunities to share best practices and network across

councils were recommended by numerous respondents. Regular

communications from the Kentucky Governor’s Office of Early Childhood to the

Page 12: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 12

public and stakeholders about the importance of early childhood was

recommended.

Opportunities for local councils to meet at the annual statewide

conference or via another forum were recommended. One respondent

recommended moving the statewide conference to May or early June since the

end of June conflicts with the school district closing of the fiscal year budget.

One respondent recommended provision of a statewide online data system to

simplify local data collection and reporting requirements. Improved monitoring

and accountability of local councils was recommended.

Specific to multi-county councils, respondents recommended a

requirement for rotation of meetings between counties, assisting counties that

want to remain independent, and additional supports to assist with getting all

partners to participate.

CECC Request for Application Reviewer Survey

Respondents

Email invitations with a link to complete an on-line survey were

successfully sent to 75 individuals who served as a CECC Request for

Application reviewer during fiscal years 2015-16 and/or 2016-17. Thirty-nine

individuals responded to the survey, a response rate of 52%. This was an

excellent response rate given that online survey response rates typically vary

from 10% to 30%.

Results

Application Processes

Review respondents replied very favorably to questions regarding the

fairness of the Request for Applications. Approximately 85% agreed or strongly

agreed that the RFA process is fair and equitable with no one responding that

they disagreed or strongly disagreed (approximately 15% neither agreed nor

Page 13: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 13

disagreed). Similar responses were given when asked if the funding process was

equitable for both new and returning CECCs (90% agreed or strongly agreed and

10% neither agreed nor disagreed).

Similarly, responses were very favorable when asked about the

transparency of the review process with approximately 79.5% agreeing or

strongly agreeing to the transparency of the process and approximately 20.5%

neither agreeing nor disagreeing. There was some slight disagreement when

asked about the clarity of the RFA. Table 4 provides a summary of responses.

Table 4. Reviewer Perceptions of the Request for Applications.

Question Strongly

Agree Agree Neither

Agree Nor Disagree

Disagree Strongly Disagree

The goals and requirements of the RFA are understandable for applicants

28% 49% 18% 2.5% 2.5%

The goals and requirements of the RFA are understandable for reviewers

36% 56.5% 5% 2.5% 0

The RFA scoring system is understandable for applicants

24% 55% 18% 3% 0

The RFA scoring system is understandable for reviewers

25.5% 72% 2.5% 0 0

Terminology used in the RFA and the scoring

system is defined and clearly understood by all

23% 54% 18% 5% 0

Respondents were also asked about the selection and training process for

reviewers. Responses to the statement that there was an appropriate application

process and skill level determination for all reviewers included 74.5% agreeing or

Page 14: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 14

strongly agreeing, 20.5% neither agreeing nor disagreeing, and 5% disagreeing.

Responses were also favorable in terms of the training provided for RFA

reviewers (79.5% agreeing or strongly agreeing, 15.5% neither agreeing nor

disagreeing, and 5% disagreeing). When asked whether the information and

materials provided to RFA reviewers were sufficient to complete the review,

responses were very favorable (97.5% agreeing or strongly agreeing and only

2.5% neither agreeing nor disagreeing).

Review process questions were also included, and respondents were in

agreement that the timeline for review was sufficient (92% agreeing or strongly

agreeing and only 8% neither agreeing nor disagreeing). Communication

between the Kentucky Governor’s Office of Early Childhood and reviewers was

also seen as good (90% agreeing or strongly agreeing, 7.5% neither agreeing nor

disagreeing, and 2.5% disagreeing).

Open-ended Questions

Two open-ended questions were included in the survey, and a summary

of responses follows.

What recommendations do you have for improving the RFA documents?

Several of the responses noted the variability in responses from one

applicant to the next and the variability in grant writing skills among councils.

Recommendations included providing a drop-down menu of choices for

particular questions, adding greater clarity to the wording and requirements for

those with less grant-writing skill, and providing online training and technical

assistance for RFA applicants.

There were also recommendations regarding the content of the RFA. A

recommendation was made to encourage a progression from year to year by

requiring applicants to summarize successes from the prior year and build a

proposal to build upon the successes. Designing the RFA in such a way that the

Page 15: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 15

reviewer can score directly on the application rather than going back and forth

between two documents was recommended.

What recommendations do you have for improving the RFA process?

Respondents noted training and technical assistance needs. One

respondent noted that there were several RFA responses where the proposed

local activities did not align with the RFA requirements relative to goals and

deliverables. For some councils, this may suggest the need for additional

training and technical assistance, or it could indicate that there is not agreement

between local and state entities regarding goals and objectives. It was also a

recommendation to require RFA applicants to provide greater explanation and

justification regarding local priorities. Several responses encouraged an earlier

release of the RFA to allow more local planning and community engagement in

the RFA process.

There were several recommendations regarding reviewer selection. A

recommendation was made to send an invitation for reviewers to all early

childhood higher education faculties. There was also a recommendation to send

out the invitation for reviewers earlier to ensure sufficient number of reviewers

and time for preparation and training. Ensuring that all reviewers are trained

and are knowledgeable of early childhood and the CECC system was

recommended.

Online submission and scoring was recommended. There was also a

recommendation to ensure that the titles and labels mirror each other on the RFA

and the reviewer scoring document to ensure more consistent and faster scoring

by the reviewer.

Interviews with CECC Leadership

In order to get more detailed information and input from CECCs and

increase confidence in evaluation findings through the use of multiple methods

Page 16: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 16

of data collection, individuals serving in CECC leadership positions were

identified. A matrix of CECCs funded during 2015-16 and/or 2016-17 was

created to identify councils by single or multiple county representation; rural or

urban designation (US Census definitions and designations of rural and urban

were used); CECC Tier I, Tier II, and Tier III funding levels; and by Kindergarten

Readiness scores (either an average county score above the state average or

below the state average). Using a random number calculator, 29 CECCs were

selected to participate in the interview sample. An analysis of how the sample fit

within the matrix described above was completed to ensure good representation

of each category. Good representation across variables was obtained.

Respondents

At least two leaders (typically the chair, co-chair, past-chair, Secretary,

Treasurer, and/or Project Coordinator) from the identified 29 CECCs were

invited to participate in an interview. A total of 35 interviews took place. Two

individuals were interviewed for 6 councils (4 single-county councils and 2

multi-county councils). One individual was interviewed for 23 councils (18

single-county councils and 5 multi-county councils).

The sample included 22 single-county CECCs and 7 multi-county CECCs.

A total of 13 urban counties and 31 rural counties were represented across these

29 councils. Seventeen of the councils in the sample were recipients of CECC

Tier III or the maximum funding level, with the remaining 12 councils receiving

CECC Tier I or II funding levels. Of all the counties represented in the sample,

24 of them had kindergarten readiness rates listed in the 2017 Early Childhood

Profile that fell above the state average. The remaining 20 counties had rates that

fell below the state average.

Page 17: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 17

Results

The responses were analyzed as a full group and as subgroups of urban

versus rural, by funding level, and single-county versus multi-county. No

significant differences were found in responses by rural/urban or single-/multi-

county. Since multi-county councils often had some counties with Kindergarten

Readiness rates above the state average and other county members with rates

below the state average, differences in responses based on this variable were not

analyzed. Funding level did seem to impact some responses. Whether the

council received the maximum level of funding or not did not appear to be the

basis for differences; rather, the total amount received appeared more predictive

of differences in responses, as highlighted in the analysis that follows.

Application Processes

Several respondents highlighted improvements in the RFA documents.

Positive comments addressed the streamlined nature of the documents and the

use of drop-down boxes. A few respondents noted the difficulty of the format in

terms of fitting all the information into prescribed boxes and columns. Concerns

were also noted regarding inconsistency of wording across sections, use of

jargon, duplication of information regarding prior council activities, and

repetitive information requests. The requirement to get letters of support from

all partners was a concern for several of the interviewees, and there were

recommendations to allow more local flexibility on this requirement and to only

require letters of commitment for new partners for councils with prior year

approved applications and funding.

Interviewees were also asked about RFA processes. Of the 29 councils

represented in the sample, 19 of them highlighted concerns with the timeline of

the Request for Application (RFA). Primary among the timeline concerns was

the lack of sufficient time for including all council members in the grant planning

and writing process, sometimes creating discord among council members

Page 18: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 18

because only a few were involved in the application. Additional concerns

related to timing included inconsistency in RFA timelines from year to year, RFA

timeline conflicts with end of school activities or start of school activities

(preventing involvement of important partners), and lack of communication

about timing and due dates. Respondents were also concerned regarding the

impact of the timing of the RFA on funding and local activities. Gaps in funding

and delays in implementation of activities were noted by several respondents.

There were several recommendations to align the RFA with the school fiscal

year. It is important to note that this year’s RFA was delayed in order to

complete this funding system evaluation; this delay likely impacted responses.

In terms of supports provided for the RFA process, two respondents

raised concerns regarding delays in response to inquiries, and lack of sufficient

training was noted by several respondents, particularly for new chairs and

relative to changes in the RFA. The lack of council staffing, dependence of

volunteers, lack of experience in grant writing among council members, and lack

of information technology support and knowledge were also concerns. A

number of interviewees recommended opportunities for sharing information

across councils during the RFA timeline and/or after awards are determined.

Another respondent recommended assigning a staff member from the Kentucky

Governor’s Office of Early Childhood as the contact person for application

assistance.

Lack of clarity regarding the review and scoring processes were concerns

for several respondents. One respondent reported that information was shared

at a state conference that funding was equally split among councils and the RFA

review and scoring had little impact on funding, negatively impacting council

member investment in the RFA. Several respondents also recommended

feedback regarding the applications or distribution of the scoring sheets.

Page 19: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 19

State Supports to Community Early Childhood Councils

The most frequent response to questions regarding needed council

supports underscored the desire for opportunities for CECC members to meet

together, receive training, and share best practices. Both statewide and regional

meetings were recommended as well as webinars and conference calls. Topics

for training and technical assistance that were identified by interviewees include

effective board development, by-laws development, team-building, non-profit

establishment, English Language Learner supports, family engagement, family

outreach, new chair training, leadership development, new All STARS system,

budgeting, and data collection and measurement.

Interviewees were asked about the role of the annual state conference in

supporting CECCs. Although there were a significant number of respondents

that spoke in positive terms regarding the conference, its focus on training for

teachers was reported as its strength. Most respondents noted that the

conference no longer offers opportunities for CECCs to meet, receive training,

and share ideas. Other concerns noted regarding the state conference included

lack of notice resulting in budget amendments, cost, and insufficient space (sold

out this year and people were turned away).

When asked about CECC participation in the School Readiness Summits,

half of the councils were unaware of or not invited to participate in the Summits.

Although several of those that did participate reported a positive impact and

good collaboration, many raised concerns regarding duplication of efforts and

recommended better coordination across school districts and CECCs. One

respondent noted that problems were created in multi-county councils when

school superintendents in some county school districts participated in the School

Readiness Summits and superintendents in other county school districts did not.

Recommendations were made to utilize the CECCs as the convener of the

Summits.

Page 20: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 20

Overall, most interviewees recommended improvements and supports

that would enable greater collaboration across CECCs. Improvements to the

Kentucky Governor’s Office of Early Childhood website were recommended,

including incorporating easy to use local directories, improved links, and more

information on local activities. More frequent newsletters and webinars were

also recommended.

When asked for input regarding potential changes or supports for local

planning, many respondents recommended opportunities for sharing best

practices and ideas across the state. A few respondents stated that a lack of

communication and support from the Kentucky Governor’s Office of Early

Childhood had a negative impact on local planning. As an example, one

respondent noted that there is no CECC directory or listserv to make contacting

colleagues in other councils easy, and another reported that multiple inquiries

were required before a response was received. Increased flexibility in

membership based on local parameters was recommended.

Respondents were split in their perceptions of communications between

the Kentucky Governor’s Office of Early Childhood and CECCs. Approximately

half of interviewees spoke positively regarding communications, and the other

half raised concerns about lack of consistent communication. Chief among

concerns was the lack of notification regarding staff changes and the lack of a

directory identifying staff responsibilities to assist with call inquires. Other

concerns include lack of timeliness in responses and lack of clarity regarding

focus of emails (e.g., information only, requires action, etc.). A few respondents

recommended local CECC visits by state staff.

Perceptions regarding the Work of Councils

Interviewees, in large part, were very positive about the work of local

councils. With a few exceptions, interviewees associated many benefits to their

communities and the state from the council system. When asked about the

Page 21: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 21

strengths of local councils, the most frequent response addressed the community

collaboration and diverse partnerships that resulted from the work, with

associated reduced duplication of services, maximization of financial resources,

and increased service access for children and families. The second most

frequent response to this question addressed the increased awareness of and

focus on school readiness in their communities. Others noted the value of

localized planning and decision-making, the ability to problem-solve and

address specific local needs, reaching hard-to-reach families, and creation of a

common vision.

Respondents, when asked about challenges associated with CECCs, most

frequently reported difficulties in getting all the partners to the table. Noting the

volunteer nature of council membership, with almost all members having full-

time jobs, interviewees discussed the difficulty in completing activities. This

appeared to be particularly problematic for smaller councils without sufficient

funding to hire staff or a project coordinator. Two respondents discussed the

negative impact on participation of having multiple local coalitions (e.g.,

Preschool Partnership, School Readiness Summit teams, CECC) with

overlapping membership and responsibilities. Many respondents included the

change in funding parameters that prohibited council support of child care

training and mini-grants as an impediment to getting child care participation on

the council. A few also noted the difficulty of getting all school districts to

participate. The difficulty of outreach to the most at-risk, hard-to-find children

and families was also a reported challenge. Other challenges that were raised

included lack of cooperation across partners, lack of resources and leverage

opportunities in rural counties, and lack of transportation.

Interviewees reported that the strongest motivation for partners to

participate as members of CECCs was their commitment to young children, their

families, and their communities. Respondents also noted that motivations were

Page 22: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 22

driven by work responsibilities as council activities were aligned to the paid

work roles of most of its members.

When questioned about the specific work of the councils, the activities

that were recounted by the majority of respondents as evidenced-based and most

effective were one-day family events where community partners gathered to

share resources, provide activities for children, and distribute information on

school readiness. Book distribution and early literacy activities, such as the Dolly

Parton Library, were the next most frequently reported activity. Child and

parent engagement opportunities were frequently noted with a range of intensity

from one night to five weeks of training. Several interviewees discussed the

distribution of resource bags or kits, often with a focus on school readiness.

Many respondents indicated that child care trainings were the most effective

activity, although they were no longer allowed. Three interviewees noted

developmental screening opportunities.

When asked about how their CECCs inform the community about the

value and importance of early childhood, most respondents listed multiple

mechanisms. Distribution of materials and resources during community and

partner events, through public service announcements and social media, at

presentations before civic and business groups, and via partner organizations

(e.g., health department, pediatricians, home visiting entities, parks, libraries,

housing authorities, early care and education providers, etc.) were frequently

noted. One interviewee talked about the power of branding in the council area

to deliver consistent messaging through the wide distribution of branded

materials.

Funding and Reporting Requirements

There were few recommendations regarding reporting requirements, with

several interviewees noting that they were relatively easy. Recommendations

that were made included release of the reporting templates earlier, consistency of

Page 23: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 23

reports from year-to-year, clearer guidance on requirements, and follow-up

feedback on annual reports. Most interviewees reported on the importance of

the Early Childhood Profile in informing their local activities (e.g., choosing

activities and resources for events and/or bags based on average scores in

various developmental domains), their reports, and the return on investment for

the state.

When asked about data collection and measurement, respondents most

frequently noted the difficulty of collecting data locally given the volunteer

structure of the councils. Most reported using counts of participation or number

of resources distributed. A few discussed pre- and post-surveys at family

engagement events. Only one council reported use of a valid and reliable

observation-based tool. Several respondents recommended that data be

collected at the state level or defined by the state. One interviewee suggested the

creation of a state database specifically for councils to enter state-identified data

points.

When asked about the funding of councils, the most frequent responses

related to the importance of continuing to fund councils and to how funds are

distributed. Interviewees from councils receiving under $10,000 were more

likely to raise the issue of low and insufficient funding amounts. These councils

were more likely to have negative views of how funding disbursement and

requirement decisions were made. Overall, many respondents noted that it is

unclear how funding decisions are made, and greater clarity on the funding

formula was recommended. The gaps in funding due to the timing of the RFA

were concerns of several respondents as well as the prohibitions on spending for

food, child care, and transportation. The majority of respondents that reported

using the budget amendment process noted its ease of use, with the only

recommendation being distribution of the requirements at the beginning of the

fiscal year or placing the application on-line.

Page 24: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 24

State-Local Partnerships – A Review of Councils in Other States

A number of states have utilized local councils or partnerships as

important components of their early care and education systems. Local councils

are often more effective than state-level entities in identifying and addressing

specific local needs and coordinating services in their communities. They also

are often more successful in building support for early childhood issues at the

local level, leveraging additional resources.

The following information provides an overview of state-local

partnerships in six states that may prompt some policy discussions and be

helpful in analyzing Kentucky’s system. Like Kentucky, four of the states are

recipients of the federal Race to the Top – Early Learning Challenge (RTT-ELC)

grant: Colorado, Maryland, North Carolina, and Oregon. In terms of the

amount of funding directed to local councils and the number of children birth to

age 5, Maryland is most similar to Kentucky. Multiple sources were used for this

information, including resources from the Early Learning Challenge Technical

Assistance Program and documents found on each of the states’ early learning

websites. A table summarizing this information is included as Appendix B.

Arizona

In 2006, Proposition 203 was passed, a citizen’s initiative to fund quality

early childhood development and health programs for children birth to age 5

through an 80-cent per pack increase on tobacco products. A statewide board,

First Things First, was established along with regional partnership councils to

share the responsibility of ensuring that funds are spent on strategies that will

improve the education and health outcomes for Arizona’s young children.

Governance Structure

The state board, First Things First, provides oversight and infrastructure

support to 28 local/regional councils. The regional partnerships are made up of

Page 25: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 25

11 volunteer members, representing parents, educators, business leaders, tribal

representatives, health professionals, philanthropists and leaders of faith

communities. Responsible for local planning and collaborative delivery of

services, they determine which early childhood services will be funded in their

communities. Goals and Priorities

First Things First, as a key partner in Arizona’s early childhood system, focuses

on evidence-based strategies designed to support school readiness. The state

board is responsible for establishing priority areas. Their priority areas include:

Increase access to high quality early care and education and children’s

readiness for school

Support stable and nurturing families

Increase access to health care

Support a coordinated, integrated, and high quality early childhood system

Ensure early childhood education and health professionals are well-prepared,

highly skilled, and well compensated

Increase public appreciation of children’s early years

Local councils decide how to best utilize funds in their communities to support

these goals and priorities.

Funding

Regional partnerships are funded with an additional tax on tobacco

products; there is no state legislative appropriation. Some private funds are

raised. First Things First administers statewide initiatives and awards regional

and statewide grants through competitive Requests for Grant Applications.

Funds are allotted to local councils based on the birth to age 5 population, which

was 436,657 young children in 2016. The total council allocation for 2016 was

$106,932,236.

Page 26: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 26

Activities

Regional partnership activities to support their state goals focus on three areas

in Arizona:

Program quality and access supports through Quality First to help regulated

programs, trainings for Family, Friend and Neighbor care; and resources for

Summer Transition to Kindergarten)

Family Support through parenting kits, Birth to 5 Helpline, Family Resource

Centers, parent classes, and home visitation

Preventative Health supports through screenings and assistance with access to

health insurance and care

Accountability

To ensure accountability at the local level, each regional partnership must

provide information that shows they are providing the services to children

required in their contracts with First Things First. They are also subject to annual

performance and fiscal monitoring and comprehensive programmatic

monitoring every 3 to 5 years. Every two years, an extensive study of local needs

and resources is required to help inform state and local improvements and

services. This data is utilized to monitor the impact of investments and services.

Ten School Readiness Indicators are used to guide and measure progress:

% of children receiving at least 6 well-child visits in first 15 months of life

% of children ages 2-4 with BMI in healthy weight range

% of children age 5 with untreated tooth decay

% of families that report they are confident and competent to support their

children

% of children kindergarten ready across developmental domains

% of children enrolled in a 3 to 5 star rated ECE program

% of children with special needs in an inclusive 3 to 5 star rated ECE program

Page 27: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 27

% of families spending no more than 10% of median regional family income on

ECE

% of children with undetected developmental delays at kindergarten entry

% of children exiting preschool special education and entering regular

education in kindergarten

Colorado

Colorado has a long history of strong local decision-making structures.

Building on local child care pilots established in 1996, Early Childhood Councils

were established in legislation in 2007.

Governance Structure

Each Early Childhood Council (ECC) in Colorado decides on its own

governance structure and must have a fiscal sponsor to receive funding. A non-

profit membership organization, ECC Leadership Alliance, represents and

supports the 31 local councils.

Goals and Priorities

The Colorado Office of Early Childhood establishes statewide early childhood

goals, which are:

Access to high quality early care and education and children’s readiness for

school

Stable and nurturing families

Access to health care

Early childhood social, emotional, and mental health promotion

Early Childhood Council priority areas are aligned to these goals, and their

system framework currently incorporates access, quality, and equity outcomes

and benchmarks. Current benchmarks focus on:

Page 28: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 28

Promoting transitions of the children in their communities – from infants to

toddler classrooms, toddler to Pre-K classrooms, and transition to kindergarten

Promoting family engagement in QRIS

Improving the quality rating of all licensed early learning programs

Funding

A combination of Child Care and Development Block Grant funds, other

state/federal grants (e.g., RTT-ELC, Infant and Toddler Quality and Availability

Program, and the Maternal, Infant, Early Childhood Home Visitation Program),

and local and private funds are used to fund local councils. Councils receive

funding on a 3-year grant cycle. The total allocation to local early childhood

councils in 2016-17 was $7,684,733. In 2016 in Colorado, there were 337,973

children birth to age 5.

Activities

Advancing programs, resources, and support around:

Early learning – professional development, school readiness, support for

special needs, and environmental support activities

Family support and parent education – information, resources, home-visiting,

leadership, self-sufficiency activities

Social, emotional, and mental health – training, screening, mental health

consultation, and support services

Health – access to prenatal and health services, insurance referral,

immunizations, training, and other support services

Accountability

Early Childhood Councils receive an annual compliance review by the

State Department based on their evaluation report. The report addresses

Page 29: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 29

program benchmarks for the identified state priority areas and incorporates

metrics based on their local strategic plan.

Iowa

An alliance of state agencies, organizations, and community partners

formed Early Childhood Iowa (ECI) in 2001.

Governance Structure

A State Board provides oversight and infrastructure support to 39 local

councils representing all 99 counties in Iowa for local planning and collaborative

delivery of services.

Goals and Priorities

Access to high quality early care and education

Children ready for school

Stable and nurturing families

Access to health care

Safe and supportive communities

Funding

Local councils are funded through State General Funds, and councils

received approximately $22,000,000 statewide in FY 2016. The birth to age 5

population in Iowa in 2016 was 200,620.

Activities

Local activities and strategies align to the 5 priority areas and include:

General health services

Dental services

Prenatal services

Screening and referral

Page 30: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 30

ECE tuition assistance and scholarships

ECE technical assistance

ECE child transportation services

Literacy supports

ECE nurse consultants

WAGE$ supports

Quality improvement supports

Professional development training, mentoring, & coaching

Crisis care services

Family support services

Resource libraries

Coordinated intake

Child seat safety checks

Public awareness activities

Accountability

All ECI areas collect performance and fiscal data on services provided and

file annual reports. Performance is evaluated via a Levels of Excellence Rating

System process. Annual report data is also compiled and shared as the State ECI

Report.

Maryland

Early Childhood Councils grew out of RTT-ELC strategic planning and

were established in legislation in 2014.

Governance Structure

A State Board provides oversight and infrastructure support to local

councils for local planning and collaborative delivery of services.

Page 31: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 31

Goals and Priorities

To establish annual benchmarks of school readiness skills of children entering

kindergarten

To conduct a periodic needs assessment of the quality and availability of ECE

services

To create an action plan for increasing participation of children in existing

programs including outreach to special and underrepresented populations

To coordinate locally the RTT-ELC goals and objectives, and other initiatives

To report progress and challenges to the State Advisory Council

Funding

Early Childhood Councils receive $1.1 million from Race to the Top-Early

Learning Challenge funds. A funding formula that took into account child (ages

birth to 5), children with disabilities, and English-Language Learner populations

was used to distribute funds across the 24 councils. The birth to age 5 population

in Maryland in 2016 was 367,003.

Activities

Local activities vary but include:

Professional development supports

Family engagement supports

Reach Out and Read partnerships with health care providers

Technical assistance and support to ECE providers participating in EXCELS,

the Maryland QRIS

Family outreach activities

Community engagement activities

Page 32: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 32

Accountability

Councils complete quarterly progress and annual evaluation reports on

their activities using results-based accountability metrics.

North Carolina

The North Carolina Partnership for Children, Inc. was created to oversee

local Smart Start partnerships in state statute in 1993, with the most recent

changes to the statute occurring in 2016.

Governance Structure

The North Carolina Partnership for Children, Inc. oversees a network of

75 non-profit organizations serving all 100 counties within North Carolina with a

focus on use of data-driven results (outlined in performance standards).

Goals and Priorities

Increase quality of child care

Strengthen families

Advance child health and development

Improve early literacy

Funding

State funding is leveraged with Federal Child Care and Development

Block Grant, local, and private funds along with local in-kind donations. $142.1

million was expended in local partnership funds in 2015-16. The population of

children birth to age 5 in 2016 was 608,802.

Activities

Provision of training and technical assistance to support increased quality of

child care

Page 33: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 33

Expansion of early literacy activities

Provision of health, nutrition, and safety supports

Provision of family education and resources through child care programs to

increase parents’ knowledge of child development

Coordination of development screenings

Facilitation of access to child care subsidies and PreK

Accountability

The North Carolina Department of Health and Human Services oversees

the performance standards for Smart Start and conducts annual contract

monitoring. NCPC also monitors local councils for adherence to performance

standards and publishes an annual report. Fiscal independent audits are

required no less than every 2 years, and third-party system evaluations are

conducted periodically.

Some of the accountability findings include:

Increase in % of children in 4 & 5 star programs (the average star rating for

children enrolled in regulated child care improved from 2.76 in 2001 to 4.31 in

2016)

Parents participating in Reach Out and Read (ROR) were 92% more likely than

new participants to engage in reading activities with their children

Increase in the number of children who received appropriate developmental

screenings (NC has highest rate of developmental screenings in the nation)

Smart Start was found to reduce special education placements in grades 3, 4, &

5 by nearly 10% and reduce a child’s chance of retention in 5th grade by 13%

Oregon

Sixteen regional/community-based Early Learning Hubs, serving 36

counties, were created in legislation in 2013.

Page 34: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 34

Governance Structure

Oregon Department of Education provides funding and support to local

councils for local planning and collaborative delivery of services, and a separate

nonprofit organization provides primary support for the councils.

Goals and Priorities

Create aligned, coordinated, and family-centered early learning system

Ensure school readiness

Ensure healthy, stable and attached families

Funding

Early Learning Hubs are funded through State General Funds. The most

recent allocation was $4.7 million allocated over a two-year period. There were

237,420 children birth to age 5 in Oregon in 2016.

Activities

Local activities focus on:

Coordination of early learning services

Alignment and integration of efforts across sectors (i.e., ECE, K-12, health,

human services, and business community)

Improved results for at-risk children

Leverage of additional private and public funds, including in-kind support

Accountability

The Department of Education Early Learning Division oversees Early

Learning Hubs and defines performance metrics. Monitoring protocols were

developed and visits began the spring of 2017.

Early Learning Hubs are required to develop a strategic plan and report on 8

metrics:

Page 35: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 35

Increase # of children attending high quality early learning programs

participating in QRIS

Improve kindergarten readiness rates

Increase # of children who receive developmental screening by age 3

Increase # of children with primary health care home

Increase # of at-risk children served across the system

Decrease the cost of service (decreasing administrative overhead)

Decrease the age of onset of services

Page 36: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 36

Conclusions and Recommendations

Conclusions

Request for Application Processes

Although both council members and reviewers were, on average, positive

regarding the RFA process, there appear to be a number of areas for

improvement. Greater transparency regarding the review and scoring process as

well as the funding formula is recommended in order to engender greater trust

and partnership between state and local leaders. Although CECC concerns with

the timeline were likely impacted by this year’s RFA delay due to this evaluation,

it remains true that consistency in the timeline and process for applying would

support better local planning and implementation of local activities. Release of

the RFA six to eight weeks prior to the due date would ensure adequate time to

bring partners together for planning and writing the application.

Council Strengths and Challenges

For those councils with strong member participation and collaboration,

there were reports of significant leveraging of resources to maximize the funding

provided by the State. Increased local awareness of the importance of school

readiness and the early years and support for early childhood issues were

important outcomes reported by council leaders. Families with young children

were provided with a number of resources, training opportunities, materials, and

activities. This evaluation confirms that local partnerships can be effective in

identifying and meeting local needs and building local commitment.

As noted in the findings, the challenge for many CECCs is engaging all

the local partners and having a strong collaborative partnership to leverage

funding and spread the work across an integrated system of local services. In a

number of areas, council planning, decision-making, and work falls on a few

shoulders, limiting the impact in terms of services provided, public awareness,

Page 37: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 37

data collection, and outcomes. The change in CECC funding parameters, away

from supporting early care and education providers and toward more outreach

and family engagement activities, identifying and supporting families with

children that are not in care, likely impacted the feedback received during this

evaluation. Many councils appear to be struggling with the change in direction.

Interestingly, there were very few significant differences among council

responses between rural/urban councils or single-/multi-county councils. It

appears that this lack of differences in responses is likely linked to funding

levels, the only variable that revealed significant differences. There are a number

of urban councils that receive funding levels similar to rural areas, as well as

single-county councils that receive amounts similar to multi-county councils

when the number of counties served is considered. The variability in responses

occurred between those councils receiving the lowest funding (under $10,000)

and those receiving more. With less funding, council responses were more likely

to be negative and the council challenges greater.

Council Impact

CECCs are an important part of Kentucky’s school readiness goal, and

their activities focus primarily on spreading the school readiness message in their

communities. Almost all of those interviewed identified commitment to children

as the major council member motivator, and the success of their work was often

noted in terms of how well they worked together with their local partners to

support young children and their families. Despite this focus of their activities,

there was a lack of clarity among council respondents regarding the priorities of

council work, and use of evidence-based practices was infrequently reported.

Often council members framed their impact in terms of how many families they

reached or how many children participated in activities, rather than changes in

knowledge, skill, or behavior.

Page 38: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 38

Data collection and measurement is difficult for the majority of councils.

Data collection plans are often limited to counts of resources distributed or

number of participants, understandable given the type of activities that are

implemented, their local approved plans, and their funding levels. Frequently

dependent on volunteer board members or staff of various partnering

organizations with other responsibilities, little appears to be collected and data

that are collected appear to lack reliability and validity. There also seemed to be

lost opportunities for obtaining data reflective of good outcomes. For example,

developmental screening activities were reported by a few of the interviewees

but collection of findings and referrals were not tracked. There were exceptions

where pre- and post- evaluations of satisfaction, and most promising, behaviors

and skills were conducted. These practices were associated with councils that

employed council or project coordinators.

Council Supports

Most CECC members regard the Kentucky Governor’s Office of Early

Childhood as a strong and supportive leader. Based on the findings of this

evaluation, however, CECCs are often operating in separate silos. There is little

communication across councils, and council members are not benefiting from

hearing about best practices among colleagues. Although there appear to have

been some resources and opportunities in the past, according to survey

responses, there are currently no state or regional meetings specifically planned

to bring together council leaders or members, no training for new chairs, no

listserv for council chairs and/or members to share information, few webinars

and training opportunities for council members, and technical assistance appears

to be limited. The perceived absence of these supports results in a lack of

cohesive direction for council work.

Page 39: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 39

Alignment of Council Activities with Kentucky School Readiness Goal

Many of the respondents reported the use of the Early Childhood Profile,

and specifically the Kindergarten Readiness Rates and Brigance scores, in their

planning of local activities. School readiness brochures, flyers, parent guides,

and supports were included in the majority of council activities. When

questioned about the value of local councils, many spoke to the number of

families that were provided with resources to support their child’s school

readiness because of council efforts. There appears to be a strong alignment of

council priorities to Kentucky’s School Readiness Goal.

Demonstration of Return on Investment

Return on investment analysis compares the economic value of the

benefits of programs and policies with their associated costs. Calculations

require measuring total costs of the program, determining and measuring the

outcomes of the program, valuing the outcomes, and comparing the benefits and

costs. Total cost must include the value of any in-kind donations, such as rent-

free office space or volunteers’ time, and a determination of any costs that would

be incurred in the absence of the program. Valuing outcomes requires making

valid comparisons of a wide range of impacts.3

The most likely outcome of council work is an impact on kindergarten

readiness rates, and the Kentucky Early Childhood Advisory Council noted the

paramount importance of local CECCs in supporting school readiness in their

last report.4 Since the establishment of a statewide Kindergarten Readiness Rate

in Kentucky, there has been a significant increase in readiness rates statewide,

from a 49% readiness rate in 2013-14 to a 51% readiness rate in 2016-17 (p < .05).

However, council efforts are one piece of a much larger system effort in support

3 Karoly, L. A. (2010). Toward standardization of benefit-cost analyses in early childhood interventions.

Arlington, VA: RAND. 4 Kentucky Governor’s Office of Early Childhood. (2015). Early childhood council bi-annual report:

Seeding Kentucky’s future. Frankfort, KY: author.

Page 40: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 40

of school readiness. Furthermore, data collected from local councils tends to

provide counts or levels of satisfaction, rather than outcomes that reflect changes

in behavior or skill, making it difficult to determine impact. There is also a lack

of data collected on in-kind donations, which may be substantial at the local

level.

In conclusion, although a formal return on investment calculation specific

to council work is not possible as part of this analysis, it appears from this

evaluation that CECCs are one contributing factor in Kentucky’s significant

increase in kindergarten readiness rates.

Recommendations

Organization of Councils

Many of the Community Early Childhood Councils in Kentucky receive

relatively small pots of funding. Spreading the current statewide CECC budget

across so many entities dilutes the potential impact of local planning and service

provision, and it prohibits many CECCs from hiring project coordinators to assist

with reporting and data collection. A reduction in the number of CECCs is

recommended.

This will not be a popular proposal in many areas, as local leaders may

fear loss of the ability to address specific county needs. To support as positive of

a transition as possible, the Kentucky Governor’s Office of Early Childhood

should provide the ‘what’ and the CECCs should provide the ‘how’. The

decision-making regarding which councils merge should be led by local councils

but should be guided by state parameters and support. These state parameters

might include designating a minimum birth to five population that must be

served by a council, counties included in the mergers must be contiguous, and

council membership requirements that ensure representation across all counties

included in a multi-county council.

Councils will need support to undertake mergers. Planning guides,

meeting facilitation technical assistance, and strategic planning resources are

Page 41: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 41

recommended. Utilizing successful multi-county councils as mentors may be

helpful.

Funding

One of the potential benefits of merging councils is that state CECC

funding can be divided between fewer entities resulting in larger pots of money.

Although the funds have to be spread across larger areas, there are efficiencies of

operation that will be realized as well as the potential for less duplication of

efforts and improved staffing.

It is also recommended that CECCs are strongly encouraged and

supported to raise local and private funds to supplement their state funding.

One recommended approach is to award additional RFA points for securing

additional funding and/or to add additional funding to performance criteria and

standards. Fiscal reporting and monitoring should incorporate local and private

funding. Training and technical assistance on securing additional funds should

be provided.

Priorities, Outcomes, and Benchmarks

The Kentucky Early Childhood Advisory Council has reported that

CECCs are a key strategy in increasing collaboration and improving local

implementation of their system work plan.5 The Kentucky Early Childhood

Advisory Council went on to note that, at their best, local councils are designed

to bring together diverse members of the community and mobilize them to work

collaboratively as change agents to achieve community level improvements in

school readiness. There is good evidence that a number of councils are operating

‘at their best’ and are achieving community level improvements that impact

readiness rates. The lack of a cohesive and consistent understanding among

5 Kentucky Governor’s Office of Early Childhood. (2015). Early childhood council bi-

annual report: Seeding Kentucky’s future. Frankfort, KY: author.

Page 42: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 42

CECC leaders regarding their role in Kentucky’s early childhood plan suggests

the need for more consistent language and alignment.

Kentucky’s system work, with its focus on high quality learning

environments, supportive families, and access to data to support school

readiness, provides a strong framework for alignment of local CECC plans.

While maintaining the overall goal of school readiness, it may be helpful to more

explicitly tie Council priorities, outcomes, and benchmarks to the Kentucky All

STARS goals and the Race to the Top-Early Learning Challenge work in CECC

communications. Although a framework for CECC work was included in the

RFA, it is important for everyone to understand how CECCs fit with the roles of

other system entities in Kentucky’s early childhood work in order to build

commitment and vision and create feelings of being “part of the team”.

For greater impact, it is recommended that the Kentucky Governor’s

Office of Early Childhood identify evidence-based practices that CECCs should

support. One focus area might be offering an introductory training to early care

and education providers regarding Kentucky All STARS, helping promote greater

participation across early care and education programs. A second recommended

focus area is facilitating evidence-based family engagement and training

opportunities, such as Strengthening Families, HANDS, or Born Learning

Academies. Expanding access to developmental screening and referral is

another area that has potential for great impact on school readiness. For

alignment with the ‘access to data’ goal, local plans can continue to address

strategies to share the Early Childhood Profiles and raise awareness of school

readiness among all community members. Performance metrics could be

identified for each of these areas.

Performance standards and benchmarks would be helpful to making local

planning more strategic and for monitoring of progress. Benchmarks could be

statewide, with input from CECCs, or could be developed by the local councils

with guidance, training, and technical assistance.

Page 43: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 43

Accountability

When funding levels are low, accountability metrics are at risk of being

overly burdensome and can take away from use of funds for the intended work.

Those making investment decisions, however, are wise to establish

accountability measures to assure funds are put to intended use, and it is

unlikely that additional funding will be directed to a system component without

strong evidence of a wise investment. CECCs have the potential to be a primary

component of Kentucky’s early childhood system and to justify additional

funding. Realizing this potential will require a stronger accountability system.

Expectations for accountability should be developed in concert with

council merger plans. Annual performance and fiscal desk monitoring are

recommended, with on-site monitoring staggered across councils every two to

three years. Reporting requirements should be aligned with performance

standards and best fiscal practices. A statewide data dashboard for CECC

benchmark reporting is recommended.

Council Supports

In addition to the training, technical assistance, and resource supports

required for council mergers, efforts should focus on creating a network of

CECCs. Currently, many council leaders feel isolated, and opportunities to not

‘reinvent the wheel’ are lost because council members are not talking to each

other and sharing best practices. Instituting biannual or quarterly meetings or

conference calls of council chairs is recommended. In addition, a listserv or

special website for sharing information and resources across councils, more

frequent training webinars, regular newsletter highlights of work in various

council areas, and designated Kentucky Governor’s Office of Early Childhood

staff technical assistance contacts are suggested.

Page 44: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 44

Change Implementation Options

Implementation of the recommendations in this report based on the

evaluation of Kentucky’s CECCs require careful creation of new implementation

design planning and development of support structures. There are fiscal,

programmatic, and policy considerations. Two options are offered. Each option

requires an integrated stage-based framework for implementation to guide the

work.

Transition Option 1

As Kentucky remains eligible for Race to the Top – Early Learning Challenge

funding, Option 1 promotes immediate change implementation in order to take

advantage of funding for infrastructure development that may not be available

once the federal funding is over. This option requires state and local work

simultaneously. Steps in this option include:

1. Identifying the metrics and requirements for council mergers based on

funding parameters, contiguous geography, and size impacts (limitations

on the number of total council members supporting more efficient council

meetings and operations must be balanced with representation across all

included counties)

2. Provision of local planning grants to support merger decisions. This step

is likely to require many councils to suspend current activities and focus

on merger planning for the remainder of the 2017-18 fiscal year. Those

councils that meet the new threshold size are provided with a second year

of current funding to continue 2016-17 activities

3. Development and distribution of merger supports, such as resource

manuals, guidelines, training, and technical assistance

Page 45: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 45

4. Framework and messaging refinement to clarify CECC mission and

priorities aligned with Kentucky All STARS and Race to the Top – Early

Learning Challenge priorities

5. Development of performance standards and benchmarks with input from

CECCs or development of training and technical assistance to support

CECC use of local performance metrics tied to the state framework and

priorities

6. Development of state infrastructure for improved CECC support,

including website and listserv, data dashboard, trainings, and technical

assistance

7. Development of an accountability plan for fiscal and performance

reporting and monitoring

8. Release of new RFA by mid-March with a submission deadline in early

May, and funding decisions announced by June 1 with a July 1 release of

funding

Transition Option 2

This option provides for a longer planning and development timeline and

provides more time for CECCs to develop merger partnerships and plans.

1. Utilize fiscal year 2017-18 for state-level planning and development,

including all the items outlined in Option 1 with the exception of #2, local

planning grants to prepare for mergers

Page 46: Commonwealth of Kentucky Community Early Childhood … Evaluation Report - 2017.pdfTable 2 provides an overview of these responses. Table 2. CECC Survey Responses regarding Training

Kentucky CECC Evaluation 46

2. CECC RFA proceeds for fiscal year 2017-18 with additional training and

technical assistance provided to support improved data collection and

measurement

3. Development of a transition plan for mergers over the next 1 to 2 years