the children's behavioral health smart model of car

39
The Children’s Behavioral Health SMART Model of Care Ensuring Access to Earliest Intervention for Hawai‘i’s High-Risk Children Birth Through Five Years Ira J. Chasnoff, MD GG Weisenfeld, EdD

Upload: hawaii-community-foundation

Post on 25-Jul-2016

214 views

Category:

Documents


1 download

DESCRIPTION

This briefing paper defines a proposed project to expand and support the diagnostic and intervention strategies available to Hawaii's youngest children prenatally exposed to alcohol and drugs.

TRANSCRIPT

Page 1: The Children's Behavioral Health SMART Model of Car

The Children’s

Behavioral Health

SMART Model of Care

Ensuring Access to Earliest Intervention for

Hawai‘i’s High-Risk Children Birth Through Five Years

Ira J. Chasnoff, MD

GG Weisenfeld, EdD

Page 2: The Children's Behavioral Health SMART Model of Car

2 | P a g e

The Children’s Behavioral Health SMART Model of Care

The purpose of this report is to describe the outcome of a one-year community-

based planning process to develop a model of behavioral health care for

Hawai‘i’s children, ensuring that all high-risk children birth to five years of age

throughout the state will have full access to early identification, comprehensive

assessment, and a full range of therapeutic services.

The SMART Model of Care has been developed through a partnership between

Hawai‘i Community Foundation and Hawai‘i’s State Departments of Health,

Human Services, and Education as well as the Zero to Three Courts, the Health

Care Transformation team in the Office of the Governor, and the Hawai‘i

Primary Care Association. The Model of Care represents the dedicated work

and leadership of these agencies.

What do we know? The early years are the most important. We know this much is true: the first

few years of children’s lives are vital for their ultimate growth and

development. During this time, the basic difficulties that define life-long mental

health, developmental, and learning problems can begin to emerge.

We also know that a child’s development is a dynamic process, involving both

social and biological factors that contribute to success or failure. From day one,

the child interacts with the surrounding environment and seeks the nurturing

support that will help him achieve full potential for growth and development.

Thus, behavioral health problems can emerge from a wide variety of biological

and environmental factors. Just as premature birth, prenatal exposure to

alcohol or drugs, or poor maternal nutrition can harm fetal brain development,

family violence, substance abuse in the family, or maternal depression can

interfere with the child’s brain development after birth. The problems that

emerge as the child enters school leave the child unprepared for learning.

Right now in Hawai‘i, children who need early intervention for developmental

and behavioral health problems are not identified until they are older; the best

opportunity to help them has been missed. Experienced professionals point

out that many children are struggling in school with problems that could have

been addressed much earlier. Others point to children who get in trouble with

the law and know that their difficulties could have been identified much earlier

in life and steps taken to help them avoid developing serious and perhaps life-

long problems.

Children with developmental or behavioral health problems who are identified

early in life and receive the services they need have a better chance to develop

to their full potential – at home, in school, as an employee and as a citizen.

However, the human benefit to the child is not the only benefit. Families benefit

when children are helped early on in life because this avoids school difficulties

and behavioral problems. Schools benefit - ask any teacher about the

Page 3: The Children's Behavioral Health SMART Model of Car

3 | P a g e

disruptions in the classroom due to children who have difficulty controlling

their behavior. The foster care system will benefit if there are fewer

placements needed to find a child a home. The State benefits from having more

children functioning well in the normal settings of life and fewer occupying

Dependency Court, Special Education classrooms and the corrections system.

The benefits are both human and financial.

“High-risk” children are those children who have exposure to risk factors that

are known to lead to problems in living. For example, children born to mothers

who used tobacco, alcohol and/or illicit drugs during pregnancy are at risk for

developing developmental and behavioral problems. Exposure to domestic

violence, poverty, neglect and abuse are other risk factors that add to the

complexity of life for many of Hawai‘i’s children. An estimated 20% of children

in the US have a diagnosable behavioral health condition, and about 2%-5%

suffer from a serious behavioral health disorder that causes substantial

impairment in functioning at home, at school, or in the community.1 There is no

reason to think that these numbers do not apply in Hawai‘i.

There are social and fiscal ramifications for society when children are not

identified early and provided appropriate care. These children typically

display more aggressive and disruptive behavior that places them on a

trajectory associated with a range of negative life outcomes in youth and early

adulthood.2 On the other hand, when children are identified as needing

behavioral health services early and receive the appropriate care, they are3:

Less likely to receive psychiatric inpatient services [savings of $1,433

per day4]

Less likely to visit an ER for behavioral and/or emotional problems

[savings of $165 per visit5]

Less likely to be arrested [savings of $4,142 per month per child6]

Less likely to repeat a grade [$10,736 per child7]

Less likely to drop out of school [$41,369 per youth8]

Less likely to be arrested as a juvenile [$5,656 per arrest9]

More likely to remain in a foster care placement [savings of $1,790 per

month per child10]

Children with serious behavioral health conditions incur costs in multiple

child-serving systems, including mental health care costs, MEDICAID, TANF,

high school drop out costs, and child welfare costs. Colorado calculates that

early intervention produces a reduction of 40% of projected future

expenditures, making the Return on Investment (ROI) $1.80 per each dollar

spent on children under the age of 511.

Hawai‘i’s children are at risk for severe problems of developmental,

behavioral, mental health, and social/ emotional functioning due to a wide

range of factors (Kids Count Data Center, 2014; HYIPR Report FY 2013,

DHS/CWS Annual Progress Report, 2014):

• 1,542 children were low birth weight (8.1% of total births)

Page 4: The Children's Behavioral Health SMART Model of Car

4 | P a g e

• 101 children died by age one (2011)

• 46% of children entering foster care entered by age 5

• 893 children were in foster care under the age of 5 in 2013

• Over 700 young children were confirmed abused/neglected in 2013;

46% were under 1 year of age

• 3.5% of infants and toddlers participated in Early Intervention [majority

for speech/language]

The planning process Responding to the issues facing children and families in Hawai‘i, Hawai‘i

Community Foundation (HCF) led the effort to develop a Model of Care for

high-risk children throughout the state. HCF organized a team of state leaders

from the public and private sectors to assess the current status of children in

Hawai‘i, examine departmental and agency approaches to a variety of factors

that impede appropriate behavioral health development, and develop an over-

arching strategy to promote the early identification and treatment of high-risk

children and their families. The team reflected a public-private partnership and

crossed organizational and professional boundaries:

Hawai‘i Department of Health

o Family Health Services Division

o Children with Special Health Needs Branch

Hawai‘i Department of Human Services

o Child Welfare Services Branch

o Med-Quest Division (Medicaid)

Hawai‘i Department of Education

Hawai‘i Zero to Three Court, Family Court, First Circuit

Health Care Transformation Project, Office of the Governor

Hawai‘i Primary Care Association.

Governance of the planning team was guided by a core set of principles agreed

to by all team members.

1. The Leadership Group:

a. is a voluntary collaboration between interested public and

private/community sector stakeholders

b. can add representatives of other stakeholders by consent of the

group

c. is not a governmental body and is not subject to Sunshine Laws

2. The Leadership Group:

a. provides leadership to advance the group’s vision and mission

b. makes recommendations to:

a. improve access to and quality of behavioral health services

to children ages 0 to 5 years old in Hawaiʻi

b. coordinate between agencies/entities to eliminate gaps

and duplication of services, leverage available funding

more effectively, and create smoother, more effective

Page 5: The Children's Behavioral Health SMART Model of Car

5 | P a g e

transitions between caregivers for children needing

services and their families.

c. holds each other accountable for forward progress to achieve

agreed goals

3. Duration:

a. Anticipated life span of the Leadership Group is 3 to 5 years,

unless terminated sooner by the members of the group

b. Frequency of the meetings and progress goals/milestones are to

be determined by the group.

4. Authority of the Leadership Group:

a. The Leadership Group has the authority to convene meetings, set

agendas, decide meeting procedure, amend these ground rules,

and terminate the existence of the Leadership Group

b. The Leadership Group has no authority to make any decisions for

any of the agencies/entities represented in the Leadership Group.

Each agency/entity retains its own independent authority to make

decisions about and implement the recommendations of the

Leadership Group.

5. Decision-making process:

a. Decisions of the Leadership Group preferably will be made by

consensus. However, the Leadership Group may decide to have

specified agenda items decided by a vote of individual members.

b. Decisions of the Leadership Group are recommendations only

and are not binding on the member agencies/entities belonging

to the Leadership Group.

c. Silence = acquiescence. In other words, members who say

nothing about an agenda item are assumed to be in agreement

with the decision of the group.

d. A written record of decisions by the Leadership Group may be

retained for internal use of the group and for individual member

organizations to understand and implement recommendations.

6. Governance structure:

a. The Leadership Group may create subcommittees or other

advisory groups to work on specific topics or tasks as directed by

the Leadership Group, consisting of designated staff from each

department or entity represented in the Leadership Group and

other experts and stakeholders by invitation.

b. Any such subcommittee or other advisory group shall be subject

to these ground rules.

c. All recommendations from all subcommittees and advisory

groups shall be subject to final review and approval by the

Leadership Group.

Page 6: The Children's Behavioral Health SMART Model of Car

6 | P a g e

7. Record-keeping and confidentiality:

a. The meeting discussions, meeting materials and notes, and

related emails and communications:

a. are for the use of members of the Leadership Group and

their respective staff

b. are not intended for publication or dissemination outside

the Leadership Group unless approved in advance by the

Leadership Group

b. Formal meeting minutes and record keeping are not required.

Each member is expected to keep his or her own records of the

meetings and materials related to the work of the Leadership

Group.

8. Role of the Hawaiʻi Community Foundation

a. To convene and facilitate meetings of the Leadership Group

b. To support the work of the Leadership Group with research and

analysis at HCF’s option, including support for the research and

advisory work provided by Dr. Ira Chasnoff and GG Weisenfeld

as contractors of HCF.

Theoretical Framework The specific aim of this initiative is to build a system of care through which all

children in Hawaiʻi birth to 5 years of age have access to the interventions and

therapeutic services they need to support the children’s achieving their full

potential.

Shared values

A core set of shared values shape the planning team’s thinking:

• Systems improvement must reflect the needs of children and their

families, be simple, and be easy to access.

• Statewide access is vital.

• The system needs to embrace cultural practices and place value on

cultural diversity and competence.

• Early Intervention, especially before the age of 6, benefits children the

most.

• Evidence-based practices provide the best opportunity for improving

outcomes of children and families.

• Financial sustainability will be built on primary and secondary payors

and on public and private sector financial support.

• The system will not be dependent on individuals within departments, but

will survive change in leadership over time.

Definitions

Based on these values, it was recognized that successful early childhood

development must be defined as broadly as possible. Behavioral health

encompasses social, emotional, behavioral, and relational development as well

Page 7: The Children's Behavioral Health SMART Model of Car

7 | P a g e

as motor, speech and language, etc. Thus, as we seek to identify and intervene

with all children at risk in Hawaiʻi, this broad definition guides our thinking.

It also was recognized that children can be found throughout the community,

not just in specific, narrowly defined settings. Thus, a second task was to take

a broad view of children’s lives in Hawaiʻi, considering the narrow gates where

children gather throughout the islands. These narrow gates include places

where medical, social service, and educational services are provided, as well

as other community-based primary sites of contact in which children can be

found.

Community-based is the preferred term for the approach taken because it

places a special emphasis on the breadth of the effort. It is not narrowly

medical, though it depends heavily on the community’s physicians and other

health care providers. It is not a public health initiative alone, though the role

of public health professionals and outreach workers is essential. It is not driven

by mental health treatment, per se, but does recognize the importance of

quality treatment capabilities throughout the community. It is not powered by

the authority of the courts, though the effort would be missing an enforcement

element if the courts were absent. Similarly, the business community, the

church community, the schools all have a stake in the success of the effort and

will play an important role. The important point is that the authority of the

approach comes from the broad base upon which it rests and in which it is

rooted.

SMART: The Core Intervention

SMART is an acronym that stands for screening, decision-making, assessment,

referral, and treatment.11 It is the “core intervention” that the planning process

sought to establish in the community.

Screen Screen all children for behavioral health risk. Largely this is

accomplished by becoming a fixed part of primary health care for

pregnant women and children’s primary health care as well as

outreach into multiple narrow gates around the community.

decision

Making

Make a decision within the primary screening site as to whether

immediate early intervention will provide the support and services

the child needs or whether the child needs to move to higher levels

of intervention.

Assess Those children who screen positive are given an appropriate level

of assessment to determine service and intervention needs.

Refer Those children who require higher levels of care are referred up

through a linked and integrated system.

Treat Those children who are referred to each level of intervention and

treatment receive quality, evidence-based treatment that is

appropriate for the child’s and family’s circumstances.

Page 8: The Children's Behavioral Health SMART Model of Car

8 | P a g e

Levers of change

One of the team’s first steps in the planning process was to identify and then

prioritize the top 3 levers of change to implement the system:

Establishing a joint goal, “vision statement” to which all are committed

Map interventions available at each level of the system, examining

issues of

o Coordination

o Accessibility

o Screening and earliest intervention

o Costs across total system

Examine and resolve barriers to data sharing

o Federal definition

o Lack of data system

o Consent process as needed

o Standardization of screening tools

Research and Investigation for Background Information The team’s work was supported by data and information collected through a

variety of surveys and investigation of existing public and private agency

services, policies, and procedures.

Site visits

In January 2015, the Hawai‘i Community Foundation sponsored site visits to

three programs developed through a similar planning process and successfully

operating SMART systems of care:

• Cradle to Crayons [Phoenix, AZ]

• MCSTART [Monterey, CA]

• Desert Mountain Children’s Center [Apple Valley, CA]

Cradle to Crayons (C2C) is housed in the Arizona County Judicial system and

serves children birth to three years in the foster care system. Three renovated

buildings connected to the courtrooms offer space for visitation, child and

family therapy, substance-abuse treatment, and early education services.

Selected court judges have received special training in early childhood

development and handling cases with children under age 3 years. With an

annual budget of approximately $2 million, the C2C Judges were able to

dismiss dependency cases on 399 children in FY 2014, with an average case

age for dismissed children during this time period being 533 days. These data

are significantly better than the overall population of the Juvenile Court Bench,

which dismissed dependencies in 282 children during that same time period

with an average case age of 907 days.

MCSTART is a children’s behavioral health clinic housed in Door to Hope, a

501 (C)(3) that initiated children’s services in 2003. Medicaid (EPSDT), child

welfare (Title IVE), substance abuse block grant monies, and private

Page 9: The Children's Behavioral Health SMART Model of Car

9 | P a g e

foundation funding support MCSTART’s public/private partnership budget of

$2.9 million. Clinical staff members, consisting of a physician, physicians’

assistant, child psychologist, 5 doctorate level therapists, an occupational

therapist, 6 parent educators, 4 case managers, and 15-20 mentor parents,

serve 400-450 children per year through a transdisciplinary approach.

Desert Mountain Children’s Center (DMCC) is a clinical services program

housed within San Bernardino County’s Department of Education and is

governed by the school district’s 15 school superintendents. Medicaid

(EPSDT), department of education, department of health (Title V) and private

foundation funding support the program’s annual budget of $2.5 million. Public

health nurses conduct outreach and screening throughout the community, and

children move through a series of increasingly complex levels of service.

DMCC providers conduct complex assessments that include evaluation of

medical, developmental, psych/social, occupational therapy and speech and

language status. Based on the assessment, recommendations are made to the

parent/guardian of the child and appropriate referrals are instituted. On site at

DMCC, a variety of treatment services are available: Individual/Family

Psychological Therapy, Parent/Child Interaction Therapy (PCIT), Theraplay®,

Play Therapy, Occupational Therapy / Sensory Processing, and Speech and

Language Therapy.

From a funding perspective across the three programs, costs/child for services

covered a wide range: $2500 to $5450. However, Medicaid EPSDT (Early

Periodic Screening, Diagnosis and Treatment) funding covered a significant

amount of this cost and can be a core source of funding for sustainable clinical

services. In addition, Title IV-E funding through the child welfare system can

be a source of funding for training of professionals and training and supporting

foster, adoptive and biologic parents. Title V federal funding through the

state’s Maternal and Child Health office provides enhanced outreach and

support services for Medicaid eligible pregnant and parenting women.

Prenatal Risk Data

Dr. Ira Chasnoff and NTI Upstream have been working in Hawai‘i to address

substance use in pregnancy since April 2007. At that time, a team of community

leaders invited Dr. Chasnoff and his team to guide the development of a

comprehensive model of prevention and intervention for families on Hawai‘i

Island. The team membership brought together representatives from various

agencies and community health centers, as well as community-based

professionals in pediatrics, early childhood education, hospital administration,

mental health, schools, substance abuse treatment, tobacco cessation, and

child protection.

As a result of that planning effort, pregnant women enrolled in prenatal care

throughout Hawai‘i Island for the past eight years have been screened with the

4P’s Plus, a validated, published screening instrument that identifies pregnant

women at risk for alcohol, tobacco, and illicit drug use. In 2012, through funding

Page 10: The Children's Behavioral Health SMART Model of Car

10 | P a g e

from the Hawai‘i Community Foundation, the screening program was

expanded to include sites on Oahu, and in 2014, outreach added Maui and

Moloka‘i. Those women with a positive screen undergo a more thorough

structured clinical intervention conducted at the same prenatal visit and

receive a brief intervention to educate them about substance use and then, as

appropriate, a referral to treatment. Widely published data have demonstrated

the success of this approach, and the federal government’s National Prevention

Task Force recommends that this strategy be utilized for all individuals enrolled

in prenatal care.

On Hawai‘i Island and Oahu, as of May of this year, 6,879 pregnant women have

been screened. The majority of the women screened are in the 19 to 34 years

age range, with 18% adolescents, and 9% of the women 35 years and older. In

response to the 4P’s Plus© screening questions, 26% of all women in the

participating sites admitted to tobacco use, 30% admitted to alcohol use, and

9% admitted to illicit drug use. Native Hawai‘ian women and Caucasian women

have the highest rates of using alcohol, and Native Hawai‘ian women have the

highest rates of tobacco use. For the entire population of women enrolled in

the study, 35% were using a substance that affects the structure and function of

the developing fetal brain.

Children’s Behavioral Health Risk

In response to multiple requests for help from Hawai‘i’s families, Dr. Chasnoff

and his team operated a free clinic on Hawai‘i Island. The purpose of this clinic

not only was to provide direct assessment and treatment services to children in

Hawai‘i, but also to try to get a first picture of the behavioral health difficulties

among children in the islands. The clinic was held on Hawai‘i Island, one week

at a time, three times per year for three years. Dr. Chasnoff brought a nurse and

a clinical psychologist from the mainland with him on a volunteer basis. Local

pediatricians provided space for the clinic in their offices at no cost. Histories

were collected on all cases referred to the clinic, and the children were

selected for assessment based on their acute need and the likelihood that the

assessment could provide them with a treatment plan that would improve their

current status.

Over 100 children were referred to Dr. Chasnoff by Hawai‘i pediatricians,

courts, and child welfare in 2011-2013. Of these children, 65 were selected to

undergo a comprehensive assessment. The children primarily came from

Oahu, Maui, and Big Island. A random sample of 22 children ranged in age

from 4 months to 16 years. Behavioral problems were the primary reason for

referral. Of the 22 children, 10 met criteria for a diagnosis within Fetal Alcohol

Spectrum Disorders, 6 had significant intellectual disabilities, 4 were

diagnosed with attention deficit hyperactivity disorder (ADHD), and 13 had a

significant mental health disorder.

None of the 22 children had ever had a full evaluation. When comparing the

multiple diagnoses that had previously been given to the children to the

diagnoses that resulted from the comprehensive assessment, 18 (82%) had

Page 11: The Children's Behavioral Health SMART Model of Car

11 | P a g e

been misdiagnosed. This high rate of misdiagnosis is consistent with the 85%

rate of misdiagnosis documented in a recent study of children in Illinois

published by Dr. Chasnoff and his colleagues in February 2015 in Pediatrics.

In examining the children’s histories, several common reasons for the high rate

of misdiagnosis emerged:

• Failure to identify risk factors, such as alcohol and drug use, in the

prenatal period. Pediatricians especially are unaware of the mother’s

use of alcohol, tobacco or illicit drugs during the pregnancy.

• Lack of children’s primary care providers’ ability to recognize risk and

know what to do for a child at risk for behavioral and mental health

disorders.

• High rate of co-occurring mental health disorders.

• Lack of assessment across all domains of child functioning.

• Limited treatment resources forced inappropriate use of medications to

address any behavioral or mental health problems.

• Multiple moves within the child welfare system resulting in constantly

changing educational placements and health care providers.

• Perceived stigma against mental health and substance abuse disorders

in families.

Survey of Hawai‘i’s Clinical Programs for Children

A survey was designed to examine the degree to which programs in Hawai‘i

are implementing the components of a SMART system of care. The survey was

distributed in January 2015 to 86 agencies across the state. Research of

websites associated with Aloha United Way Search 211; Hawai‘i Department of

Human Services and Department of Health; Children, Adult, Mental Health

Division (CAMHD) grantees; Early Intervention providers; and Home Visiting

contractors provided the list of potential participants. In addition, each of the

six Head Start/Early Head Start grantees and the 14 FQHC providers were sent

a survey. Programs were emailed a weblink that would connect the programs

to the on-line survey. Confidentiality of specific programs was ensured.

After the initial response, follow up phone calls and emails were sent to

programs to answer any questions and remind them to complete the survey.

For several programs, the data were collected during a phone conversation

and manual entries were made.

Sample Forty-six surveys were completed. Nine of the programs

responded that they served children over the age of 8 and were therefore

excluded from the sample. The remaining 37 respondents were sorted; seven

more were excluded from further analysis: 3 because they do not provide

direct services but serve as advocacy agencies and four because they had

incomplete surveys. The final sample that was analyzed was comprised of 30

agencies.

Page 12: The Children's Behavioral Health SMART Model of Car

12 | P a g e

These 30 agencies represent a wide range of programs that varied in the

number of children age birth to eight served annually [10 to 5000], the number

of sites at which services were offered [1 site to 38 locations], and the overall

size of the agency. The respondents ranged from statewide, very large multi-

service agencies that served a large number of zip codes to smaller therapeutic

settings that served one zip code. Collectively, the 30 agencies serve over

25,000 children (birth-8) annually.

Methodology and data analysis All of the programs’ responses were scored

based on the degree to which they implemented SMART components. There is

no viable methodology for measuring “decision making” through a survey

approach, so only the action components (screening, assessment, referral,

treatment) were included in the programs’ evaluation. Each of the four action

steps was given one of three scores: a blank rating [they did not provide this

service]; a lower case letter: they provided this service but not to the degree

necessary for a high-quality SMART system; or an upper case letter: they

provided the component necessary for SMART quality (Table 1). Dr.

Weisenfeld assigned these codes to each program in the sample based on the

self-reported data submitted by the survey responder.

Table 1. Scoring System for Clinical Program Survey

Did not

provide this

component

Provides this component

but not to the degree

necessary for SMART

quality

Provides this component

necessary for SMART

quality

Screening No rating Rating: s

Screens children, but

did not use a valid or

reliable tool

Rating: S

Screens children using a

valid and reliable tool.

Assessment No rating Rating: a

Assesses children but

does not include a

clinical interview,

and/or instruments,

and/or multi-

disciplinary team.

Rating: A

Assesses children using a

clinical interview, and/or

instruments, and/or

multi-disciplinary team.

Referral No rating Rating: r

Makes referrals to

programs, but does not

have a system in place

to track child’s entry into

the program.

Rating: R

Makes referrals and has a

procedure for tracking

child’s entry into the

program.

Treatment No rating Rating: t

Provides treatment but

does not use published

evidenced-based

therapies.

Rating: T

Provides treatment using

published evidenced-

based therapies.

Page 13: The Children's Behavioral Health SMART Model of Car

13 | P a g e

After the programs were assigned a code, the surveys and results were

reviewed and verified by Dr. Chasnoff. Any disagreement between the two

coders was resolved through consensus.

Results The following table summarizes the data obtained through the

survey for the 30 programs.

Do not provide

this component

Provides this

component but

not to the degree

necessary for

SMART quality

Provides this

component

necessary for

SMART quality

Screening 8 7 15

Assessment 12 14 4

Referral 8 13 9

Treatment 10 13 7

Screening 22 programs responded that they screen children. The

most common tools used were Ages & Stages-III (ASQ) and Ages and Stages-

Social Emotional (ASQ-SE). However, not all children within the programs

are screened; rather, the programs select which children will be screened,

often after a clinical assessment. Formal screening instruments were more

likely to be implemented in center-based programs, such as Head Start and

family child interaction programs (FCIL)12 programs, whereas in federally

qualified Community Health Centers, a clinical interview approach more

typically is used to screen the children. Statewide, the estimated number of

children screened is about 30%.

It should be noted that in the past five years numerous statewide workgroups

have been organized to address children’s developmental screening:

• Action Strategy: Team 3 (Governor’s Office)

• American Academy of Pediatrics: Building Bridges

• Child Mental Health Initiative (DOH)

• Early Childhood Comprehensive Systems: Screening Management

Team (DOH)

• EPSDT Workgroup (DHS)

• Hawai‘i Child Welfare Services-Program Improvement Plan (PIP2)

Steering Committee (DHS)

• Hawai‘i Wrap Services Project Coordinating Committee (DHS)

• Healthy Child Care Hawai‘i (DOH, UH)

• Hui Kupa`a Collective Impact (Governor’s Office & Phocused)

• Maternal Child Health Workforce Development (DOH)

• Screening and Assessment Workgroup (CWSB, DHS)

• Title V Screening Work Group (DOH)

Page 14: The Children's Behavioral Health SMART Model of Car

14 | P a g e

Most of these workgroups identified valid screening tools to be used for

screening; however, different tools were selected by different workgroups. For

those recommending the same instrument, there is not a consistent cut-off score

used to determine when children need services, nor are there common

protocols or policies for screening and referring children who are positive. In

addition, children often are screened multiple times. For example, when

children leave IDEA Part C [Early Intervention, DOH] and enter IDEA Part B

[SPED, DOE] they are re-screened. When children are screened in Head Start,

they are re-screened by the pediatrician, sometimes using the same tool. This

results in a significant amount of money and energy being used up by

screening efforts, depleting resources for needed assessments and treatment.

The lack of a unified data system or even simple protocols for communication

across departments and systems leads to this inefficiency and ineffectiveness.

Assessment There is no central definition of what an assessment means.

Therefore, many groups state they are assessing children, but they are actually

using screening instruments. What assessments occur tend to be focused on

the particular expertise of the provider.

Referral Referrals from one agency to another appear to be

occurring; however, there is no system to track children to ensure that they

follow through on the referral and subsequently receive the appropriate and

defined treatment identified in their assessment plan. As one provider stated

in the survey, “Our challenge is that we have no tracking, nor metrics to

provide details on this population.”

Treatment Most programs that provide treatment are not using

evidence-based models. A limited number of providers are using evidence

based treatment approaches, but in many cases they are using them with

populations for which the program was never intended. Less than 25% of the

programs we surveyed offered high-quality treatment, and these programs

only serve 739 children collectively.

Survey of State Departments and Agencies

In order to better understand the clinical landscape in Hawai‘i, each member

of the Leadership Group was asked to identify all the “narrow gates” through

which children ages 0-5 pass within their department/agency. In addition to

identifying the narrow gates, a template was provided to collect specific

information about what if any child screening occurs at this narrow gate. If

developmental screening does occur, addition information was requested:

Professional background, discipline of personnel conducting the

screening

Number of children screened per year

Instruments or approaches used for screening

Other organizations that might assist in the screenings

Geographic location of screenings

Training and professional developmental opportunities for screeners

Page 15: The Children's Behavioral Health SMART Model of Car

15 | P a g e

Quality assurance methods

Monitoring of contractors

Follow up, including brief interventions, provided to children who

screen positive

Tracking system for screening and referral data

Ten “narrow gates” were identified across the agencies and departments;

however one of the gates was a system effort that does not directly serve

children(*):

Child Welfare Services [Department of Human Services]

Med-Quest (Medicaid) [Department of Human Services]

Family Court, 1st Circuit [State Judiciary]

Special Education Section [Department of Education]

Executive Office on Early Learning [Department of Education]

Early Intervention [Department of Health]

Early Childhood Comprehensive Systems [Department of Health]*

Home Visiting Unit [Department of Health]

Child, Adolescent, Mental Health Division [Department of Health]

West Hawai‘i Community Health Center [Federally Qualified Health

Center]

Analysis of the data collected through the survey indicated that among the

responding agencies administered by Leadership Group participants,

approximately 90,000 children birth to age five living in Hawai‘i pass through a

“narrow gate” and have an opportunity for screening. The “narrow gate”

through which the largest percentage of children pass is Med-Quest

(Medicaid). Other gates see much fewer children:

Division/Program/Office Number of Children (0-

5) who Enter this

Narrow Gate Annually

% of Children (0-5) in

Hawai‘i

who Enter this Narrow Gate

Child Welfare Services 1068 (FY 2014) 1.2%

Med-Quest (Medicaid) 50,700 56.3%

Family Court, 1st Circuit 20-30 .02%

Special Education Section 11 .01%

Executive Office on Early

Learning

420 .47%

Early Intervention 3324 3.7%

Home Visiting Unit 603 .67%

Child, Adolescent, Mental Health

Division

65 .07%

West Hawai‘i Community Health

Center

Not available Not available

Some of the Divisions/Programs/Offices assess or evaluate children, but in

terms of screening children for behavioral health, only five

Divisions/Programs/Offices screen children, and these are conducted either

Page 16: The Children's Behavioral Health SMART Model of Car

16 | P a g e

by contractors or by staff [as noted]:

Division/Program/Office Group

Conducting

Screening

Number of Children

[0-5] Screened

Annually at this

Narrow gate

% of Children

(0-5) in Hawai‘i

who are

Screened at

this Narrow

Gate

Child Welfare Services Contractors Not available Not available

Family Court, 1st Circuit Staff 20-30 .02%

Home Visiting Unit Contractors Only parents are

screened for risk

factors

---

Med-Quest (Medicaid) Contractors 41,700 46.3%

West Hawai‘i Community

Health Center

Staff 26 .03%

Analysis of the survey data revealed that there are a number of opportunities

for identifying behavioral health risk in children across multiple agencies and

departments. Child welfare services (CWS) has access to all children entering

the system as well as children who remain in the custody of their families but

under CWS supervision. The system relies heavily on primary care providers

to conduct screening and referral, but there is no tracking system that ensures

that this is happening.

The courts have protocols for screening all children, but this “rarely” occurs.

Although capable of screening, it appears that the federally qualified

Community Health Centers screen very few children, and CAMHD and Head

Start, with access to numerous children, do not conduct screenings. It also is

obvious that the public is not aware of access to screening. For example, the

Department of Education’s “Operation Search” had only 11 families call in to

request screening for their children ages 3 to 5 years in the 2014-2015 school

year.

For the two programs in which staff members conduct screening of children,

there was a range of professional qualifications required. One program [Family

Court] requires a Master’s Degree in Psychology or coursework completed for

a PsyD degree. West Hawai‘i FQHC utilizes front desk staff who had a brief

training in administering the questionnaires. For the programs that use

contractors, Med-Quest and CWS rely on medical professionals [i.e.,

physicians, APRNs, and PAs]. Follow up protocols and procedures for those

children who screen positive are unclear and appear to be fairly inconsistent.

The survey also demonstrated opportunities for developing protocols across

systems. Multiple different screening instruments are used across the various

departments and programs, including some that actually are designed for

assessment rather than screening. The mix of screening instruments allows for

identification of different markers across the population and makes it difficult

Page 17: The Children's Behavioral Health SMART Model of Car

17 | P a g e

to develop any cohesive idea of what behavioral health difficulties children in

Hawai‘i are facing.

Training on screening is sorely needed. This training should include how to

provide guidance on decision-making as to the next best step for the child and

how to provide earliest intervention within the primary screening site. The

training programs should be followed up with quality assurance activities.

There is very little active review in any programs to assess the consistency and

quality of screening activities.

The survey regarding screening also revealed that there are strong early

intervention services available in the state, primarily through the Department

of Health’s Early Intervention programs for children birth to 3 years and the

Department of Education’s prekindergarten special education services for

children 3 to 5 years.

No matter what narrow gate they may enter, tracking behavioral health data for

children is problematic. Programs are not aware if any prior screening

occurred. Some programs, however, did have information systems that allowed

them to track children internally within their department/program/division,

such as Early Intervention, Home Visiting, DOE’s eCSSS system, CAMHD’s EHR

system, and West Hawai‘i FQHC’s excel worksheet database. None of these

tracking systems communicate with any other.

SMART System of Care Utilizing the data and information gathered through the planning process, the

Leadership Team came to consensus around developing a SMART System of

Care for children birth to 5 years throughout the state of Hawai‘i. As

described previously in this document, the SMART System of Care is a coordinated and integrated system of health and behavioral health care for

all children. It is grounded in13: • Screening

• decision Making

• Assessment

• Referral

• Treatment

Planning is dedicated to a shared vision for the children’s behavioral health

system of care:

Hawai'i's children will be born healthy and will thrive physically,

socially, and emotionally, supported by safe, nurturing families and

an integrated system of universal behavioral health screening,

assessment, referral and treatment that will promote the ultimate

well-being of all children and their success in school.

The central aim of the of SMART System of Care is to identify all children who

are at risk for medical, mental health, emotional, developmental or learning

Page 18: The Children's Behavioral Health SMART Model of Car

18 | P a g e

problems and ensure they and their families receive the appropriate level of

assessment and treatment they need for the children to succeed in school and

in life. The overall structure of the system can be defined through a four-tiered

pyramid based on a response to intervention model:

Tier 1 – All children receive Tier 1 interventions through

universal screening of pregnant women and screening of all children. This

tier is grounded in universal public and professional education that

addresses children’s behavioral health, including reducing the stigma

associated with mental health difficulties. Classroom teachers, health care

providers, and others who interact with children and families provide

earliest interventions and supports.

Tier 2 – Based on universal developmental and behavioral health

screening, children identified as “at risk” via a positive screen and for

whom Tier 1 interventions are not supportive enough receive Tier II early

intervention. These interventions, including family support and guidance,

can occur in the medical offices, the classroom, or through IDEA services

(Part C for children 0-3 years, Part B for children 3-5 years).

Tier 3 – Children who are not making adequate progress through

Tier 2 interventions or children whose family environment cannot support

positive behavioral and mental health development will move into Tier 3

interventions. Tier 3 interventions are grounded in targeted assessments

and address areas of deficit through group interventions, parenting

education and support, and school-based intensive instruction, specific to

the child’s area(s) of need. Specialized clinical programs in the

community, specialists in the specific area of deficit or risk, and the

schools provide Tier 3 interventions.

Tier 4 – Children who are not making adequate progress through

Tier 3 interventions or children with complex needs, including those

whose family environment does not function in a way to promote healthy

development, will receive Tier 4 interventions. Tier 4 interventions are

grounded in a comprehensive assessment across all domains of child and

family functioning that results in a wide-ranging and comprehensive

Tier 4

2-4% of children

Tier 3

6-8% of children

Tier 2

10% of children

Tier 1

80% of children

Page 19: The Children's Behavioral Health SMART Model of Car

19 | P a g e

treatment plan that can guide interventions across multiple domains. The

comprehensive assessments are conducted and the treatment plans are

developed within a highly trained, multidisciplinary children’s

neurodevelopmental health center. Therapy is provided through this

center or through community-based providers with special, high quality

expertise.

The following descriptions provide more detailed information for each tier

and how children will move through the system of care.

Tier 1

Description

All children receive Tier 1 interventions through universal screening of

pregnant women and screening of all children birth to 5 years. This tier is

grounded in universal public and professional education that addresses

children’s behavioral health, including reducing the stigma associated with

mental health difficulties. Classroom teachers, health care providers, and

others who interact with children and families provide interventions and

supports.

Goal 1A for Tier 1: Professionals and the general public will understand the

importance and impact of behavioral health problems in children. Outcomes:

1. Through a public health campaign, 75% of the general public will be

able to communicate the importance and impact of behavioral health

problems in children.

2. Through professional education, 75% of clinicians across a broad

spectrum of disciplines will:

a. Demonstrate a statistically significant increase in knowledge

regarding the prevention of and interventions for children’s

behavioral health problems.

b. Be able to communicate the importance and impact of

behavioral health problems in children.

Goal 1B for Tier 1: Women’s prenatal care providers and clinic personnel will

effectively screen and identify all pregnant women at risk for alcohol,

tobacco, and illicit drug use. Outcomes:

1. 75% of women’s and children’s health care providers and clinic

personnel in each of the islands will participate in the training and

technical assistance activities of this initiative.

2. At the completion of training activities, 75% of participants will

demonstrate a statistically significant increase in knowledge regarding

alcohol, tobacco, and illicit drug use in pregnancy and its impact on the

child.

Goal 1C for Tier 1: Pregnant women using alcohol, tobacco, and illicit drugs

Page 20: The Children's Behavioral Health SMART Model of Car

20 | P a g e

will be identified in the prenatal care setting and receive an appropriate level

of intervention. Measurable objectives:

1. 90% of women enrolled in prenatal health care will be screened for

alcohol, tobacco and illicit drug use.

2. 90% of women who have a positive screen for alcohol, tobacco, and

illicit drug use will receive a brief intervention in the prenatal health

care setting.

3. 90% of pregnant women who require further evaluation and

treatment will receive a referral to an appropriate treatment

program.

Goal ID for Tier 1: Children at risk for behavioral health problems will be

identified in a variety of clinical, social service, and educational settings and

receive an appropriate level of intervention. Outcomes:

1. 90% of children enrolled in children’s health care, social service and

educational services will be screened for risk of behavioral health

problems.

2. 90% of children who have a positive screen for risk of behavioral health

problems will receive site-based early intervention services.

3. 90% of children identified through early intervention services but who

need further evaluation and intervention will be referred to Tier 2

services.

Funding Sources to Achieve this Goal

Currently used sources of funding to cover these costs, including screening,

workforce development, parent education, and advocacy:

Head Start [Federal funds distributed to non-profits in the state] (FY2015

federal grant to non-profits is $20,035,000, this includes all HS services,

not just screenings)

Title V [Child Health] (Block grant for FY2015 is $2,144,047, this includes

all activities)

Maternal Infant Early Childhood Home Visiting [MIECHV/ACA] (FY2015

$1,000,000 and an additional $8,430,783 of discretionary funds; there is

also a state match)

Early Childhood Comprehensive Systems Grant [Federal grant through

the Health Resources Services Administration (HRSA)] ($140,000)

Title IV-B [Child Welfare Service Branch, DHS] (FY2015 is $2,032,258,

this includes all CWSB activities)

Aloha United Way’s Developmental Screening Programs (FY2015

$105,181)

Project LAUNCH [short term SAMHSA grant]

Currently unused, or slightly used, but potential sources of funding to cover

these costs, including screening, workforce development, parent education,

and advocacy:

Page 21: The Children's Behavioral Health SMART Model of Car

21 | P a g e

Title XIX, EPSDT ($119,757,247; there is an additional state match)

Private Insurance payers

Child Care Development Fund [CCDF] ($4,971,630, primarily pays for

child care)

Preschool Development Grant ($14,000,000 2015-2018)

Early Head Start/Child Care Grant (FY2016 $1,400,000)

Screening for and treating maternal depression through Title V Maternal

Child Health Block Grant (MCHBG)

Activities that Contribute to this Goal

Public health campaign addressing children’s behavioral health risk

and interventions

Professional education addressing children’s behavioral health risk

and interventions

Screening pregnant women for substance/toxic exposure

Providing a brief intervention to pregnant women

Conducting behavioral health screenings on children birth to 5 years

Storing child-level data on screenings

Sharing data from screenings with referral resources

Training those conducting screenings

Convening workgroups to discuss screening policy

Educating families on conducting screenings

Hawai‘i’s Progress on Meeting Goal

Screening, brief intervention and treatment for pregnant women using

alcohol, tobacco and illicit drugs

The majority of federally funded Community Health Centers, some

private practices, and a variety of public health and social service

agencies are screening pregnant women for alcohol, tobacco and illicit

drugs, providing a brief intervention on-site, and referring to treatment

as needed. Current data regarding substance use among the

approximately 7,000 women who have been screened were presented

previously in this document.

Conducting developmental screenings on children [before age 5]

Statewide, the current estimated number of children screened is about

30%.

Screening typically is conducted by:

Primary Care Providers: pediatricians, pediatric specialists, family

physicians, community health centers, general practice, internal

medicine

Head Starts/Preschools/Child Care Providers/Home Visitors

The tools commonly used [and the groups using them] are:

Page 22: The Children's Behavioral Health SMART Model of Car

22 | P a g e

Ages & Stages Questionnaire; Ages & Stages Questionnaire: Social

Emotional [ASQ, ASQ-SE]: Home Visitors [DOH and DHS], Child

Care/Preschool providers, Head Start

Parents’ Evaluation of Developmental Status [PEDS]: Hawai‘i

Pediatricians

Rapid Assessment Instrument (RAI) for children 4-18 [CWSB, DHS]

Child Behavioral Checklist (CBCL) for children 2-3 [CWSB, DHS]

In SFY2012, 72.2% of children eligible for EPSDT screenings were

screened.

DHS administrative rules for child care (family child care and center-

based) have an Integration of Mental Health Concepts item, which

requires that the provider regularly communicate with the

parents/guardians about the child's development and that the provider

be aware of community resources to help recognize and foster age

appropriate behavioral development in children and share the

information of community resources with the parents/guardians.

However, there is not a requirement that the provider conducts

developmental screenings.

Storing child-level data on screenings

There have been some attempts to store child-level data, but only in

certain communities, both of which are located on Oahu.

Training those conducting screenings

Numerous training opportunities occur that at times cover

developmental screenings, including using ASQ and ASQ-SE. Some

examples of these include:

Family Child Interaction Learning (FCIL) programs conduct own staff

trainings or contact DOH for support

Family, Friend, and Neighbor (FFN) training through Learning to Grow

(DHS, CCDF Quality dollars) funding

Hawai‘i Association for the Education of Young Children (HAEYC)’s

annual conference

Hawai‘i Home Visiting Network

Head Start annual training

The Parent Line (this contract requires contractors to provide training of

trainers on ASQ)

Convening workgroups to discuss screening policy

The following groups have been meeting in the past five years to discuss

developmental screening policy:

• Action Strategy: Team 3 (Governor’s Office)

• American Academy of Pediatrics: Building Bridges

• Child Mental Health Initiative (DOH)

Page 23: The Children's Behavioral Health SMART Model of Car

23 | P a g e

• Early Childhood Comprehensive Systems: Screening Management

Team (DOH)

• Hawai‘i Child Welfare Services-Program Improvement Plan (PIP2)

Steering Committee (DHS)

• Hawai‘i Wrap Services Project Coordinating Committee (DHS)

• Healthy Child Care Hawai‘i (DOH, UH)

• Hui Kupa`a Collective Impact (Governor’s Office)

• Maternal Child Health Workforce Development (DOH)

• Screening and Assessment Workgroup (CWSB, DHS)

• Title V Screening Work Group (DOH)

Next Steps

1. Increase the number of children screened:

Support pediatricians by helping problem solve some of the common

barriers they identify: a lack of time to administer screens during health

visits; inadequate compensation; lack of training in the use of specific

tools; and lack of, or perceived lack of, assessment and treatment

resources14.

Help EPSDT and private insurance plans in using the existing procedure

code (CPT code 96110) for enhanced reimbursement to providers for

using a validated tool to perform developmental screening.

Review the Medicaid state plan in terms of allowable locations for

screenings, the qualifications of who can bill for screening children, what

specific services are individually reimbursed by Medicaid, allowable

frequency of screening reimbursement, and the amount paid for each

service.

2. Align developmental screening policies:

Determine the state agency or entity that will be the decision maker for

policy decisions. One department could then order and sell the kits and

offers trainings to providers [currently this is being done by DOH for the

newborn metabolic screenings with the hospitals]

Coordinate and possibly combine some of the workgroups that have

been discussing developmental screening.

Collectively identify the valid screening tools that will be used

statewide.

For those using the same tools, identify a consistent cut-off score used to

determine when children need services.

Include all funding experts (EPSDT or private insurance payers) in the

workgroups

Develop common protocols or policies for screening and administering

the tool and referring children who are positive

Incorporate developmental monitoring into DHS child care licensing

requirements.

Page 24: The Children's Behavioral Health SMART Model of Car

24 | P a g e

3. Support workforce development:

Advocate for the regular use of developmental screening tools in the

care of pediatric patients be incorporated into training programs for

medical residents, pediatric nurse practitioners, MORE

Developing a statewide training plan for selected tool[s], possibly

working with CCDF training funds [DHS], supporting providers in

conducting workshops on talking to families about it, and understanding

what to do with results

Working with ASQ/Brookes and other publishers if identified to see if

there is a way to develop a certification/license [like first aid]

Tier 2

Description

Based on universal developmental and behavioral health screening, children

identified as “at risk” via a positive screen and for whom Tier 1 interventions

are not supportive enough receive Tier II early intervention. These

interventions, including family support and guidance, can occur in the medical

offices, the classroom, or through IDEA services (Part C for children 0-3 years,

Part B for children 3-5 years).

Goal for Tier 2: All children who need early intervention services will have

access to appropriate services through a variety of public and private agencies. Outcomes:

1. 90% of children identified as requiring Tier 2 assessment will receive an

appropriate assessment.

2. 90% of children whose assessment indicates a need for early

intervention services will receive appropriate services.

3. 90% of children who do not demonstrate progress in Tier 2 services will

be referred to Tier 3 services.

Funding Sources to Achieve this Goal

The primary source of funds currently being used to pay for the services within

this tier come from the US Department of Education’s Individuals with

Disabilities Education Act [IDEA]. Early Intervention (EI), Part C covers

children up to the age of 3 ($2,148,926 FY15) and Special Education Preschool,

Part B children aged 3 to 5 ($903,031; state match $257,701).

There is not a sliding scale in the state regulations for the EI program. There is

a clause in the statute that says that if children are eligible for Part C, the private

insurance companies do not have to pay for these services.

Activities that Contribute to this Goal

Appropriate and accurate interpretation of screening information

Referring children to early intervention with relevant data

Informing families/community about EI/SPED services

Training EI/SPED staff

Providing direct EI/SPED services to children

Page 25: The Children's Behavioral Health SMART Model of Car

25 | P a g e

Receiving, storing, and sharing data on services provided with other

providers

Hawai‘i’s Progress on Meeting Goal

Appropriate and accurate interpretation of screening information

The degree to which this is occurring cannot be determined.

Referring children to early intervention with relevant data

Early Intervention referrals primarily come from physicians.

The intake data system at DOH is being updated; currently it does not

ask if Home Visitors refer children, even though they have been. So

there may be more pathways that are being utilized to refer children,

however there is not any data to verify this.

Each agency has a different cut-off for making referrals [even if though

they mainly use the ASQ-SE].

Informing families/community about EI/SPED services

Hawai‘i’s Department of Health operates Hawai‘i Keiki Information

Service System [H-KISS] for up to age 3 and The Parent Line for children 0-

20.

Hawai‘i Department of Education’s Operation Search [ages 3-20].15

The Hilopa‘a Family to Family Health Information Center provides free

information and assistance to families and professionals on referrals,

technical assistance and training.16

The Parent Line

Training EI/SPED staff

In a recent State Systemic Improvement Plan (SSIP) Provider Survey17:

One in four staff report that they are not comfortable in understanding

age-appropriate social-emotional skills for children [birth to age 6]

63% report waning more training on communicating about sensitive

issues to the cultural stigma related to mental health services

Respondents wanted training on challenging behaviors [73%], social-

emotional development [57%], evidenced based practices in social-

emotional development [72%]

In Hawai‘i, there is a need for more speech, language, and physical therapists.

One of the barriers that has been identified by DOH is the procedure that DHRD

implements takes a long time approving positions, so there is a delay in hiring

people.

Providing high quality, direct EI/SPED services to children

Quality of Services

Early Intervention

Page 26: The Children's Behavioral Health SMART Model of Car

26 | P a g e

Hawai‘i has a lower percentage of children with a substantial increased rate of

growth compared to the nation in social-emotional development and

knowledge and skills.18

About 3 in 10 children with military insurance and about 2 in 10 children with

Medicaid [QUEST] did not achieve age-level functioning, compared to about 1

in 10 children with private insurance. 19

Almost 9 in 10 children eligible for EI services due to biological risk maintained

or achieved age-level functioning. More children will developmental delays did not age-level functioning (22%). Almost 4 out of 10 children eligible due

to biological risk and developmental delay were not functioning on par with

their peers at exit from EI. 20

Special Education Preschool

Of those children who entered the program below age expectations in each of

the following outcome, the percent who substantially increased their rate of

growth by the time they turned six years of age or exited the program in the

outcome of21:

Positive social-emotional skills (96.5%)

Acquisition and use of knowledge and skills (97.6%)

Use of appropriate behaviors to meet their needs (95.1)

The percent of children who were functioning within age expectations in each

of the following outcomes by the time they turned six years of age or exited the

program:

Positive social-emotional skills (51.2%)

Acquisition and use of knowledge and skills (51.1%)

Use of appropriate behaviors to meet their needs (60.0%)

Service Delivery

Almost 4% of Hawai‘i’s infants and toddlers receive Part C, Early

Intervention services22 and 4.9% receive services under Part B, Special

Education Preschool23.

Early Intervention in Hawai‘i does not implement a medical model. They

are a primary source provider through a coaching model. Typically, the

therapists go into a child’s home for 25-30 hours per week.

Unfortunately, the parent sometimes views this service as “babysitting.”

19% of the children referred for EI are deemed ineligible.

Children who receive EI services do so through private providers who

have state contracts [85%] and through state provided services [15%].24

There is a lack of community sites to support serving children in natural

environments when families do not want providers in their homes.

Transition from Part C to Part B

Part C [operated by DOH] serves children up to age 3, and then they

Page 27: The Children's Behavioral Health SMART Model of Car

27 | P a g e

transition to Part B [operated by DOE]. There are several mechanisms

in place to encourage a smooth transition and continuation of services

for children:

An MOA/MOU is in the process of being finalized by the DOH and DOE.

There is a STEPS State team and local STEPS teams, which bring together

Part C and Part B. This worked successfully years ago in aligning

policies, offering joint training, supporting families and children through

transitions of care, but has faced challenges. The challenges identified:

No DOE person who is solely responsible for 619. In the past DOE

had this position, the current person has lots of other responsibilities.

619 Coordinators [Part B, DOE] are not meeting as a team anymore.

No longer are the Part C and Part B attending joint trainings.

DOE does not accept Part C evaluations

Receiving, storing, and sharing data on services provided with other

providers

Early Intervention uses two different data systems one which stores

children’s social-emotional levels [as measured by the BDI-2] and

another that stores the Child Outcomes Summary [COS] rating. Both of

these systems are non-web-based. The COS ratings are entered into the

Hawai‘i Early Intervention Data System (HEIDS) which is intended to

“improve service delivery for children, assist providers in managing

their programs, and provide Hawai‘i Part C administration with data for

the purpose of assessing compliance with federal and state reporting

requirements.” 25

Next steps

1. Increase access to appropriate early intervention services:

Assess availability of programs

Educate front line screeners as to programs available and how to make

referrals

Educate families as to early intervention programs, their purpose, and

their availability

Promote self-referral to early intervention programs.

2. Align early intervention eligibility guidelines and policies:

Determine eligibility criteria and assess gaps in services that may exist

due to these criteria

Educate professionals and families as to the eligibility criteria for various

early intervention programs

Page 28: The Children's Behavioral Health SMART Model of Car

28 | P a g e

Expand, when possible, eligibility criteria beyond purely

developmental or educational issues to include the broader range of

behavioral health difficulties faced by young children

Coordinate access efforts with work groups and organizations

addressing universal screening

Ensure payment pathways for early intervention services

Require that all children birth to age 3 entering the foster care system be

assessed thru IDEA Part C Early Intervention program.

3. Support workforce development

Educate early intervention professionals as to broader implications of

behavioral health challenges beyond current organizational definitions

Developing a statewide training plan for families and early intervention

specialists

Tier 3 Description

Children who are not making adequate progress through Tier 2 interventions

or children whose family environment cannot support positive behavioral and

mental health development will receive Tier 3 interventions. Interventions are

grounded in targeted assessments and address areas of deficit through group

interventions, parenting education and support, and school-based intensive

instruction, specific to the child’s area(s) of need. Specialized clinical

programs in the community, specialists in the specific area of deficit or risk,

and the schools provide Tier 3 interventions.

Goal for Tier 3: All children who need focused therapeutic services will have

access to appropriate services through a variety of public and private agencies.

Outcomes:

1. 90% of children identified as requiring Tier 3 assessment will receive

an appropriate assessment.

2. 90% of children whose assessment indicates a need for focused

therapeutic services will receive appropriate services.

1. 90% of children who do not demonstrate progress in Tier 3 services

will be referred to Tier 4 services.

Funding Sources to Achieve this Goal

EPSDT should be the primary source to pay for funding of clinical services for

children within this level. The required state match is 50%.

There are other sources of funding that support some Tier 3 efforts including:

Enhanced Healthy Starts is a Home Visiting program with active Child

Welfare Service Branch [CWSB] paid for by TANF funds [$2,800,000,

annually].

Page 29: The Children's Behavioral Health SMART Model of Car

29 | P a g e

Title IV-B2 service categories for specific populations, primarily focused

on neighbor islands and in rural communities. [SFY2013 $946,084].

Hawai‘i has chosen for its Title IV-E waiver to address using data to

inform practice and re-entry into foster care.

Currently unused but potential sources of funding to cover these costs, include:

Title IV-E funding can be a source for training of professionals and

training support of foster, adoptive and biologic parents.

Title XIX federal funds provides an opportunity to enhance outreach and

support services for Medicaid eligible pregnant and parenting women.

Local foundations.

Title XIX to pay for public health nurses to provide care coordination.

Finance early childhood mental health consultation with CCDF funds

[thru quality dollars’, by transferring TANF funds to CCDF or the Social

Services Block Grant [SSBG], with Medicaid/EPSDT [for individual

children].

Activities that Contribute to this Goal

Ways that activities/projects/programs support children’s Tier 3 targeted

assessments and interventions before age 5:

Conducting a targeted assessment of child and family functioning that

results in a treatment plan that can guide interventions targeted at the

child’s specific deficits and challenges.

Receiving, storing, and sharing data on services provided with other

providers

A referral system will ensure that all children will have access and be

linked to the appropriate community based services indicated in the

treatment plan

Providing therapy through community-based providers with special,

high quality expertise

Hawai‘i’s Progress on Meeting Goal

Conducting a targeted assessment of child and family functioning that results

in a treatment plan that can guide interventions targeted at the child’s specific

deficits and challenges.

Statewide, there is no central definition of what an assessment means.

Therefore, many groups state they are assessing children, but these

assessments tend to be focused on the expertise of the providers.

Assessment is limited to the knowledge and capabilities of the

workforce.

The group that appears to be consistently providing targeted

assessment for children is the federally qualified health care centers

[FQHCs]. The FQHCs only serve children who are Medicaid eligible.

There are a few providers that assess children, but their numbers are

quite small [12-75 children] and tend to be focused on specific

populations, such as families experiencing domestic violence and/or

Page 30: The Children's Behavioral Health SMART Model of Car

30 | P a g e

homelessness. Some of the programs that conduct assessments

reported having waiting lists for children needing to be assessed.

Receiving, storing, and sharing data on services provided through other

providers.

DHS has continued to rely on and make minor changes to its automated

computer data system, Child Protective Service System (CPSS). The

system is used for readily identifying the status, demographic

characteristics, location, and placement goals of every child who is in

foster care. In addition, they have continued to use the SHAKA (State of

Hawai‘i Automated Keiki Assistance) system for certain functions, such

as the 48-hour Tracker to monitor timely response to child abuse and

neglect intakes, the National Youth in Transition Database, worker visit

surveys, tracking completion of transition plans for youth who will be

aging out of foster care, applications for higher education allowances

and education and training vouchers, etc. There have been work groups

and agreements to support CPSS and SHAKA in working together to

develop an effective and user-friendly way to track worker visits with

children.

A referral system will ensure that all children will have access and be linked

to the appropriate community based services indicated in the treatment plan.

Referrals are happening, however there is no system to track these

children to ensure that they receive the appropriate and defined

treatment identified in their assessment plan. One provider commented,

“Our challenge is that we have no tracking, nor metrics to provide details

on this population.”

Again, the group that reports that they have a system in place is the

FQHCs.

Several non-profits report that they have internal systems to track if

families follow-up on their treatment plans. These tend to be more

informal and based on the relationship the program staff have with the

families.

Providing therapy through community-based providers with special, high

quality expertise.

Most programs that provide treatment are not using evidence-based

models.

A limited number of providers are using evidence based treatment

approaches, but in many cases they are using them with populations for

which the program was never intended. Less than 25% of the programs

surveyed offered high-quality treatment, and these programs only serve

739 children collectively.

The Child and Adolescent Mental Health Division’s (CAMDH)

population’s age distribution is heavily skewed, with the largest

proportion of youth served being older, average age 14.1 and the

Page 31: The Children's Behavioral Health SMART Model of Car

31 | P a g e

Developmental Disabilities Division (DDD) only providers services to

adults.

Next Steps

1. Expand payment options

Support adequate provider payment rates through using a rate

methodology that reflects actual market rates to establish service

reimbursement rates.

Provide support so programs can navigate rules so they can access

EPSDT to pay for services.

The EPSDT required match comes from State funds that are typically

allocated to where the services originate [mostly DHS and DOH]. DOE

is a potential source for additional matched funds.

2. Support DHS/DOH in reviewing and possibly expanding their contract

requirements so that:

Contracted providers have financial incentives and/or sanctions based

on quality of care indicators, and specified outcomes at the system,

program, and child and family levels.

There are funds to carry out contract monitoring activities and reporting.

3. Unify referral process/system

Develop web-based centralized referral system

4. Enhance quality of services

Train the workforce to be certified on various interventions/therapies.

This includes ongoing quality assurance evaluations.

Tier 4

Description

Children who are not making adequate progress through Tier 3 interventions

or children with complex needs, including those whose family environment

does not function in a way to promote healthy development, will receive Tier 4

interventions. Tier 4 interventions are grounded in a comprehensive

assessment across all domains of child and family functioning that results in a

wide-ranging and comprehensive treatment plan that can guide interventions

across multiple domains. The comprehensive assessments are conducted and

the treatment plans are developed within a highly trained, multidisciplinary

children’s neurodevelopmental health center. Therapy is provided through

this center or through community-based providers with special, high quality

expertise.

Goal for Tier 4: All children with complex needs who require comprehensive

neurodevelopmental assessment and therapeutic services will have access to

appropriate services through a central children’s behavioral health center. Outcomes:

Page 32: The Children's Behavioral Health SMART Model of Car

32 | P a g e

1. 90% of children identified as requiring Tier 4 comprehensive

neurodevelopmental assessment will receive an appropriate

assessment.

2. 90% of children whose assessment indicates a need for

comprehensive neurodevelopmental therapeutic services will

receive appropriate services.

Funding Sources to Achieve this Goal

Currently, there is not an agency or organization operating at this Tier. Once

one is identified, the funds needed to operate these services will include:

Some of the organization or agency’s current operating funds for indirect

or overhead costs.

EPSDT funds, which would require the agency or organization has a

system for billing Medicaid.

Private insurance payers, which also requires a system for billing.

Foundation support.

It is critical to remember that this model of care is only sustainable if multiple

sources of revenue are committed to its success. Moreover, the sources of

revenue must make an enduring commitment to avoid instability.

Activities that Contribute to this Goal

Ways that activities/projects/programs support children’s Tier 4

comprehensive neurodevelopmental assessments and interventions

before age 5:

Conducting comprehensive assessments across all domains of child

and family functioning that results in a wide-ranging and comprehensive

treatment plan that can guide interventions across multiple domains.

These assessments include Pediatricians, Public Health Nurses, Infant

Mental Health specialists and other trained professionals for health and

behavioral health problems that will result in a long-term treatment plan.

Receiving, storing, and sharing data on services provided with other

providers

Providing therapy in a centralized location or in collaboration with

community-based providers with special, high quality expertise

Training a multidisciplinary team to implement high-quality behavioral

health services

Hawai‘i’s Progress on Meeting Goal

Conducting comprehensive assessments across all domains of child and

family functioning that results in a wide-ranging and comprehensive treatment

plan that can guide interventions across multiple domains for children who

require this level of care. These assessments include Pediatricians, Public

Health Nurses, Infant Mental Health specialists and other trained professionals

Page 33: The Children's Behavioral Health SMART Model of Car

33 | P a g e

for health and behavioral health problems that will result in a long-term

treatment plan.

There is no formal effort to identify and monitor high-risk children across

all aspects of the Hawai‘i Island community.

There is no single site responsible for ensuring comprehensive

assessment for these children, although a thorough assessment must

guide effective treatment planning.

Receiving, storing, and sharing data on services provided with other

providers

This is not happening in a systematic or uniform way.

Providing therapy in a centralized location or in collaboration with community-

based providers with special, high quality expertise

Service delivery, especially treatment, is based on funding streams, so

that children at risk for health and behavioral health problems are

referred to a variety of providers through specific programs that address

a single need.

Although collaborative efforts have been attempted in the past, there is

little communication among caseworkers in the various agencies.

The school system is marginalized in the overall effort to bring mental

health services to young children. This is not seen as a core

responsibility of the schools’ special education programs except in rare

instances.

Children in families that suffer from substance abuse and domestic

violence live on the periphery of social, medical, and educational

settings so that systems of care never reach them until their complete

failure brings them to the attention of the school system, the juvenile

justice system, or child protective services.

Training a multidisciplinary team to implement high-quality behavioral health

services

To achieve this goal, training must address clinical, administrative, and

procedural approaches to comprehensive and integrated assessment

and treatment.

There is no recognized resource in Hawai‘i for providing this advanced

training.

Next Steps

Identify an agency or agencies that will create a children’s behavioral health

center for Hawai‘i. This center will provide a place where all aspects of a

child’s behavioral health, including emotional, developmental, and biological

health, can be assessed by a multi-disciplinary team and then nurtured to

support the child’s ability to function in every day life, his or her concept of self,

and the ability to relate to others.

Page 34: The Children's Behavioral Health SMART Model of Car

34 | P a g e

The first six months will be dedicated to start up, with a focus on administrative

organization, identifying and moving into the center offices, purchasing of

supplies and equipment, hiring, and finalizing clinical protocols and service

plans. In addition, as part of the start-up, we will develop the system for billing

Medicaid and other third party payers for services, including private

insurance.

The second six months will be a pilot phase in which children and families will

be seen for services at the center. Protocols will be tested to ensure

appropriate application to Hawai‘i’s unique populations are developed, data

management will be tested, and insurance and third party payer billing

procedures will be tested and finalized.

During the first year of operations, training of local professionals will begin.

Over time, local professionals will assume responsibility for the delivery of

assessment and treatment services.

Advancing the Plan: 2016 As we move into the next year, the “next step” specific strategies presented for

each of the four tiers will guide the next phase of work. Overall, the focus will

be on implementing screening and earliest interventions among early adapter

sites, enhancing the quality of services provided by Tier 3 community

programs, development of the Tier 4 Children’s Behavioral Health Center, and

linking the participating systems.

1. There are several groups currently providing some form of screening of

children at the primary contact level. We need to identify specific groups

who will serve as “early adapters” and develop appropriate Memoranda

of Understanding and data sharing agreements with these groups. One

component of the “early adapter” program will be recruiting physicians

and other health care personnel to participate in the program. Training to

upgrade the quality and consistency of screening and earliest intervention

services will be necessary.

2. Training and professional development is an acute need. It has been the

general consensus of the Leadership Team that training needs lie at the

heart of systems improvement. However, discussions of training have

been nebulous and indirect. When one examines the Tier 1 surveys, it can

be seen that quite a bit of “training” already occurs. However, it does not

appear that it always is of the highest quality, and much of it is peer to peer,

which lacks in quality control. In the early part of the year it is

recommended that the HCF team:

determine precise training needs of each department and division,

most likely via survey, interviews;

determine precise training needs of private agencies throughout

the state, especially those agencies that would provide Tier 3

services;

decide what aspects of training should be supported by the

foundation and which aspects should be left to the departments and

Page 35: The Children's Behavioral Health SMART Model of Car

35 | P a g e

private agencies;

develop a training of trainer strategy for any training we will

conduct/support so as to make the impact of the training

sustainable;

develop a training schedule, including appropriate faculty.

This will position us in the latter half of the year to begin training in an

organized and meaningful way that is linked to roll out of the system.

3. The administrative and clinical development of the Tier 4 Children’s

Behavioral Health Center will necessarily be a long and slow process that

should be initiated in the early part of the year. This should include site

visits to mainland programs that have been developed through the SMART

system strategy. Other members of the leadership team could well benefit

from the exposure, also.

4. Evaluation, both process and outcome, will be the factor that drives

sustainability and ongoing funding of the system. Good evaluators want to

be involved in the early stages of planning so as to understand the

underpinnings of the program and expectations for outcomes. Thus, we

should bring an evaluation team in relatively soon and start laying out how

we want to document and track change, including cost/benefit analyses.

5. A communication strategy needs to be developed in order to stay ahead of

public expectations. This strategy will need to include action steps with

target audience(s), a timeline, and the development of appropriate on-line

and hard copy print materials for distribution. We might consider the

development of a web site dedicated to the SMART system of care, which

could be used to communicate with professionals as well as the public.

1 U.S. Department of Health and Human Services. (1999). Mental health: A

report of the Surgeon General. Rockville, MD: U.S. Department of Health and

Human Services, Substance Abuse and Mental Health Services Administration,

Center for Mental Health Services, National Institutes of Health, National

Institute of Mental Health.

Friedman, R., Katz-Leavy, J., Manderscheid, R., & Sondheimer, D. (1998).

Prevalence of serious emotional disturbance in children and adolescents. An

update. In R. W. Manderscheid & M. J. Sonnenschein (Eds.), Mental health,

United States, 1998 (HHS Publication No. SMA99-3285) (pp. 100-112).

Washington, DC: U.S. Government Printing Office.

Perou, R. (2013, May). Mental health surveillance among children: United

States, 2005-2011. Center for Disease Control and Prevention Morbidity and

Mortality Weekly Report (MMWR).

Page 36: The Children's Behavioral Health SMART Model of Car

36 | P a g e

2 Huang, L., Stroul, B., Friedman, R., Mrazek, P., Friesen, B., Pires, S., &

Mayberg, S. (2005). Transforming mental health care for children and their

families. American Psychologist, 60(6), 615-627.

Clark, H. B., Deschenes, N., Sieler, D., Green, M. E., White, G., & Sondheimer,

D. L. (2008). Services for youth in in transition to adulthood systems of care. In

B. A. Stroul & G. M. Blau (Eds.), The systems of care handbook: Transforming

mental health services for children, youth, and families (pp. 517-543). Baltimore,

MD: Paul H. Brookes.

Cocozza, J. J., Skowyra, K. R., Burrell, J. L., Dollard, T. P., & Scales, J. P. (2008)

Services for youth in the juvenile system in systems of care. In B. A. Stroul &

G. M. Blau (Eds.), The systems of care handbook: Transforming mental health

services for children, youth, and families (pp. 573-593). Baltimore, MD: Paul H.

Brookes.

Epstein, M. H., Nelson, J. R., Trout, A. L., & Mooney, P. (2005). Achievement

and emotional disturbance: Academic status and intervention research. In M.

H. Epstein, K. Kutash, & A. J. Duchnowski (Eds.), Outcomes for children and

youth with emotional and behavioral disorders and their families: Programs and

evaluation best practices (2nd ed., pp. 451-477). Austin, TX: PRO-ED.

National alliance on Mental Illness. (2010, July). Facts on children’s mental

health in America. Retrieved on May 31, 2015:

http://www2.nami.org/Template.cfm?Section=federal_and_state_policy_legis

lation&template=/ContentManagement/ContentDisplay.cfm&ContentID=4380

4

Pullmann, M. D., Kerbs, J., Koroloff, N., Veach-White, E., Gaylor, R., & Sieler,

D. (2006). Juvenile offenders with mental health needs: Reducing recidivism

using wraparound. Crime and Delinquency, 52(3), 375-397.

Wagner, M., & Cameto, R. (2004). The characteristics, experiences, and

outcomes of youth with emotional disturbances. NLTS2 Data Brief, 3(2).

Retrieved on May 31, 2015:

http://www.ncset.org/publications/viewdesc.asp?id=1687

3 These cost estimates are based on research that analyzes cost savings when

systems of care are implemented nationally and within sates and

communities. Hawaii-specific data are not available.

4 Stroul, B., Pires, S., Boyce, S., Krivelyova, A., & Walrath, C., (2014). Return on

investment in systems of care for children with behavioral health challenges.

Washington, DC: Georgetown University Center for Child and Human

Development, National Technical Assistance Center for Children’s Mental

Health.

Page 37: The Children's Behavioral Health SMART Model of Car

37 | P a g e

Their analysis was based on data obtained from: Agency for Healthcare

Research and Quality. (2013). Emergency room services: Mean and median

expenses per person with expense and distribution of expenses by source of

payment: United States, 2009. Medical Expenditure Panel Survey household

component data.

5 Stroul, B., Pires, S., Boyce, S., Krivelyova, A., & Walrath, C., (2014). Return on

investment in systems of care for children with behavioral health challenges.

Washington, DC: Georgetown University Center for Child and Human

Development, National Technical Assistance Center for Children’s Mental

Health.

6 National Center on Addiction and Substance Abuse (CASA) at Columbia

University. (2004). Criminal neglect: Substance abuse, juvenile justice and the

children left behind. New York: Author.

7 Cornman, S. Q. (2013) Revenues and expenditures for public elementary and

secondary education: School year 2010-11 (Fiscal Year 2011) (NCES 2013-342).

Washington, DC: U. S. Department of Education, Institute of Education

Sciences, National Center for Education Statistics.

8 Sum, A., Khatiwada, I., & McLaughlin, J. (2009). The consequences of

dropping out of high school: joblessness and jailing for high school dropouts

and the high cost for taxpayers (Paper 23). Boston, MA: Northeastern

University, Center for Labor Market Studies.

Carnevale, A. P., Rose, S. J., & Cheah, B. (2011). The college payoff: Education,

occupations, lifetime earnings. Washington, DC: Georgetown University,

Center on Education and the Workforce.

9 National Center on Addiction and Substance Abuse (CASA) at Columbia

University. (2004). Criminal neglect: Substance abuse, juvenile justice and the

children left behind. New York: Author.

10 Texas Department of State Health Services. (2011). Coordinated funding for

children with serious emotional disturbance: Current funding, services and

recommendations. Austin, TX: Texas Mental Health Transformation Working

Group, Children and Adolescent Workgroup, Children’s Coordinated

Funding Committee.

11 Gould, M. (2000). Mental health early intervention program for young

children cost of failure study. Denver, CO: Colorado Department of Human

Services.

Heilbrunn, J.Z. (2010). The cost of services revisited: Kid Connects mental health

consultation as a cost savings investment strategy. Denver, CO: Colorado

Department of Human Services.

Page 38: The Children's Behavioral Health SMART Model of Car

38 | P a g e

12 Family Child Interaction Learning (FCIL) programs delivery model involves

parent/adult family member participation and child learning time. The

programs require that the adults attend the program with the child.

13 McGourty, R. & Chasnoff, I. (2003). Power Beyond Measure. Chicago: NTI

Upstream, 2003.

14 Sices, L., Feudtner, C., McLaughlin, J., Drotar, D., &Williams, M. (2004).

How do primary care physicians manage children with possible

developmental delays? A national survey with an experimental design.

Pediatrics, 113(2):274-82.

15 To view the Operation Search brochure go to:

http://www.hawaiipublicschools.org/DOE%20Forms/Special%20Education/

Operation_Search.pdf

16 For more information about Hilopaa visit:

http://www.hilopaa.org/Pages/default.aspx

17 Hawaii Department of Health (April 1, 2015). Hawaii Part C FFY 2013

SSP/APR indicator 11: State systematic improvement plan (SSIP), phase 1.

Retrieved on 6/22/2015:

http://health.hawaii.gov/eis/files/2013/05/HawaiiPartCSSIP-PhaseI-

April2015.pdf

18 Hawaii Department of Health (June 2014). Part C early intervention, state

systemic improvement plan: Supporting infants and toddlers social emotional

development. Retrieved on 6/22/2015:

http://health.hawaii.gov/eis/files/2013/05/SSIPBrief-June2014.pdf

19 Hawaii Department of Health (April 1, 2015). Hawaii Part C FFY 2013

SSP/APR indicator 11: State systematic improvement plan (SSIP), phase 1.

Retrieved on 6/22/2015:

http://health.hawaii.gov/eis/files/2013/05/HawaiiPartCSSIP-PhaseI-

April2015.pdf

20 Ibid.

21 These are preschool outcomes for FFY 2012. They are a state-selected data

source. Sampling of children for assessment is allowed. Sample must yield

valid and reliable data and must be representative of the population sampled.

Retrieved on 6/22/2015:

http://www2.ed.gov/fund/data/report/idea/partbspap/2014/hi-acc-

statedatadisplay-12-13.pdf

Page 39: The Children's Behavioral Health SMART Model of Car

39 | P a g e

22 Early Intervention services offered include: Assistive Technology;

Audiology; Care Coordination; Family Support/education; Health; Nursing;

Nutrition; Occupational therapy; Parent-to-parent support; Physical therapy;

Psychological support; Speech and language therapy; Social work

(counseling); Specialized teaching; Transportation; and Vision.

23 For 2012-2013 SY, retrieved on 6/22/2015:

http://www2.ed.gov/fund/data/report/idea/partbspap/2014/hi-acc-

statedatadisplay-12-13.pdf

24 The private service providers include: Easter Seals; IMUA Family Services;

Ikaika Infant Toddler Development; Kau Child Development Program; Kona

Child Development Program; Waianae Child Development Program; and

North Hawaii Child Development Program.

25 Hawaii Department of Health (April 1, 2015). Hawaii Part C FFY 2013

SSP/APR indicator 11: State systematic improvement plan (SSIP), phase 1.

Retrieved on 6/22/2015:

http://health.hawaii.gov/eis/files/2013/05/HawaiiPartCSSIP-PhaseI-

April2015.pdf