seta mcqi: implementation guide

SETA MCQI: IMPLEMENTATION GUIDE

SETA Head Start CFS QA Unit 1

Table of Contents I. GOALS AND OBJECTIVES .......................................................................................................... 2

II. MONITORING FOR COMPLIANCE AND QUALITY IMPROVEMENT (MCQI) FRAMEWORK ............ 2

III. TOOLS AND METHODOLOGY .................................................................................................... 3

V. REPORTING PROCEDURES AND CORRECTIVE ACTION PLANS (CAP) .......................................... 8

VI. QA Support and Resources ...................................................................................................... 9

APPENDICES ....................................................................................................................................... 10

Appendix A – Unannounced Safety and Supervision Form and Procedure .............................. 10

Appendix B – Child Plus Checklist Summary List ..................................................................... 12

Appendix C – MCQI Checklists Flow Chart by Program Type ................................................... 13


2 SETA Head Start CFS QA Unit

I. GOALS AND OBJECTIVES

In accordance with 45 CFR Subpart J 1302.102 (b)-(d), monitoring for continuing quality improvement is conducted to:

1. Ensure procedures and program activities are implemented to comply with all federal, state and local regulations;

2. Support the program in its effort to increase staff awareness of all requirements and characteristics of high quality early childhood programs;

3. Provide actionable data and information to: (a) correct quality and compliance issues, (b) improve processes, (c) support training and professional development activities, and (d) inform leadership decisions in achieving program’s performance goals and outcomes.

II. MONITORING FOR COMPLIANCE AND QUALITY IMPROVEMENT (MCQI) FRAMEWORK

Annual Quality Assurance Monitoring Reviews

(Comprehensive or Differential Review)

Facilities Inspection

File and Document Review

Observation/Interviews

Staff and/or Parent Interview

Year-round Unannounced Safety and

Supervision Visits

Observation

Staff Interview

Quantitative and Qualitative Data Reports

(Classroom/ Site level and Program level)

Corrective Action Plans (CAP) and Time-based Closeout

Immediate Health and Safety Issues (Class/Site)

Systemic Issues (Program level)

Countywide Data Analysis, Reporting and

Development of Program Improvement

Plans/Strategies



III. TOOLS AND METHODOLOGY

ChildPlus Internal Monitoring Module (IMM) is used to implement an electronic system to schedule reviews, organize collected monitoring data and closeout identified compliance and quality issues through Corrective Action Plans (CAP).

Monitoring for Compliance and Quality Improvement (MCQI) Checklists: Program Service Area Performance Indicators are organized into checklists. There are multiple indicator checklists designed to monitor a particular program service area. Data collection methods will include onsite inspection, observation, file and document review and staff/parent interviews and ChildPlus desk audit. Monitors enter monitoring results (Compliant/Non-Compliant/Not Applicable) electronically into IMM at time of onsite or desk review.

A. Grantee Monitoring Activities

1. Year-round Unannounced Safety and Supervision Visits: Visits are conducted by QA staff and internal program staff at all classes per quarter.

2. Annual Quality Assurance Monitoring Reviews: Each program year, SETA CFS QA Unit will conduct a scheduled and coordinated review with each Delegate/Partner Agency and SETA-Operated Program (SOP).

3. Based on past and current performance of each agency, SETA allows for a Comprehensive Review (CR) or a Differential Monitoring Review (DR), a risk-based approach type of monitoring. Comprehensive Review will include monitoring all program service areas whereas Differential Review will only focus on targeted program service area(s). A differential approach provides flexibility for SETA (Grantee) to focus on areas that require more targeted monitoring, follow-up and support for continuing program improvement.

4. Within a 2-year cycle and regardless of performance level, each agency will be assigned at least one Comprehensive Review type of monitoring. SETA CFS Management and QA Monitoring Unit will have final decision on the type of monitoring to be conducted for each agency.

5. If an agency program is assigned a Differential Monitoring Review, it will be given an option to decline it and request a comprehensive review.

B. Reporting, Follow-Up and Closeout

1. Year-round Unannounced Safety and Supervision Visits: Quarterly summary reports are produced by QA Unit based on site visit results submitted by QA Unit staff and by internal program staff. Non-compliance issues identified during QA Unit-conducted visits are required to be addressed onsite (either correct the problem or start an action to resolve the problem) and are reported within 24-48 hours to SETA CFS Management and respective Delegate/Partner Administrator. Non-compliance issues identified by



internal program staff during their unannounced visits are addressed following their respective agency policy and procedures.

2. Annual Quality Assurance Monitoring Reviews:

Results and Site-level Corrective Action Plans (CAP) of Immediate Health and Safety Issues: Using ChildPlus IMM system, a Corrective Action Plan (CAP) with itemized list of non-compliances is generated per classroom/center or school site and submitted to designated agency staff. A due date and targeted resolution date will be assigned for each CAP based on safety or priority risk level.

Each agency is required to appoint and assign an overall administrative-level staff and additional program staff responsible and authorized to receive and closeout the electronic CAP from ChildPlus IMM. Only authorized staff will be given access to the ChildPlus IMM.

Results and Program-level Corrective Action Plan for Systemic Issues: A QA Program Summary Report highlighting strengths, trends and areas of non-compliance will be submitted to CFS Management and Delegate/Partner Agency administrators. A separate Corrective Action Plan will be required to address program-level issues.

IV. MONITORING PROCEDURES FOR ANNUAL MCQI REVIEWS

A. Planning and Coordination

1. Notice of Review Schedule/Calendar (Year-at-A-Glance): At the beginning of each program year, Delegate/Partner agencies and SOP will receive notification of targeted month and type of review it will be receiving for the year.

Comprehensive Review (CR) Criteria:

Program has gone through major changes in staff assignments (teaching staff and organizational support) the prior year.

Past QA Monitoring Review Compliance rates of 85% and lower in 3 of 4 major areas of program services namely: (1) Health, Nutrition and Safe Environments; (2) Family, Parent and Community Engagement, (3) Education, Disabilities and Mental Health, and (4) ERSEA.

Differential Review (DR) Criteria:

Past QA Monitoring Reviews indicate consistently low compliance (85% and below) in certain focus areas but consistently high (90% and more) in other areas within a program service area. SETA QA Unit can decide to monitor only the focus areas that are consistently low in compliance.



The agency participates in other monitoring reviews (e.g., CDE/State Review; CACFP) and

quality improvement activities (e.g. QRIS, CLASS) and there is demonstrated evidence of

satisfactory performance.

Differential Review can be based on the service area or site/class selection to be monitored.

2. Planning Meeting between QA Unit and Agency Leadership Team: Within 30-45 days prior to the scheduled review, formal communication and an in-person team planning meeting with agency leadership team will be scheduled to decide the scope and logistics of the review process. Each agency leadership team is provided an opportunity to provide input on the focus area of the monitoring review. A program’s areas of strengths, concerns and progress on prior non-compliance and corrective action plan will be considered. Depending on focus area, agency will be required to submit necessary documents (e.g. CLASS scores, etc.) for review. CFS Management and QA Unit will make the final decision on:

Site selection

Child file selection

Scope and focus of the review (Program Service Area checklists)

Duration of review dates and Corrective Action Plan process

3. Site Selection and File Reviews -Sampling Procedures: Selection of classes/centers or school sites will be based on several factors, and depending whether a program receives a comprehensive review or a differential review. Center-based Option (including EHS-CCP)

During a comprehensive review, a “randomizer” application will be used to select classes/classrooms and school sites or centers. The final set of participating classes and school sites centers will be a mix of options in a center-based model

CASE EXAMPLE #1: Health Services Program Services: If mandated health screenings focus

area was consistently 85% and below in the past 3 years of monitoring but follow-up

services on needed health/medical/nutrition concerns focus area was consistently 85% and

higher, the former will be monitored but the latter may be waived for the program year.

CASE EXAMPLE #1: If a program has provided written evidence that several of its classes have recently completed a successful CDE (State) review in the area of educational services, a different set of classes could be considered for the educational services monitoring. Alternatively, focus area checklists to monitor other aspects of educational services (just pertaining to Head Start requirements) may be considered for those classes that had a recent successful CDE review.

CASE EXAMPLE #2: If a program has provided written evidence that several of its classes have received strong CLASS scores, focus area checklists designed to monitor center-based child-adult interactions may be skipped for these classes.



(i.e. mix of part-day, full-day, double session). No less than 10% of classes/centers or school sites will comprise a sample. Center-based and home-based program options are monitored separately.

During a comprehensive review, no less than 10% of children’s files will be monitored. The same children’s files will be reviewed across focus areas to get a comprehensive picture of services provided to the child and family. File review will include ChildPlus desk audit and child file review.

During a differential review, final selection of classes/centers or school sites will be based on agency’s past monitoring review results and their current performance, combined with input provided by agency leadership team during the planning process. Depending on the overall objective of the monitoring review, sample size will change.

During a differential review, sample size of files will vary (more or less than 10%) depending on the overall objective of the monitoring review

Home-based Option

All service areas will be reviewed regardless of comprehensive review and differential review due to the comprehensive approach of the home-based option.

Sample size will be based on size of funded enrollment, to be selected by number of Home Visitors (1 Home Visitor= 12 families caseload).

4. Electronic Submission of Pre-Review Documents: During the planning phase (30-45 days prior to the targeted review date), agency will be required to submit pertinent information and documents such as:

Updated site and class directories (to be verified with HSES)

Written Policies and Procedures

Recruitment Plans

School or class calendars/field trip and socialization schedule

Agency guide to accessing electronic systems used such as ChildPlus, attendance tracking (sign in/out system), etc.

If the program uses a multi-media platform for assessing and recording DRDP assessments and Individualization (e.g. Learning Genie, DRDP Tech), an information session with agency staff for QA monitoring staff will be requested and scheduled. QA staff should be provided appropriate access to the system.

B. Onsite Review, Duration of Review and Monitoring Activities

1. Entrance Interview or Group Staff Interview: An Entrance Interview will be arranged in-person or via conference call at the beginning of the review period. In certain situations, a group staff interview may be held in lieu of an Entrance Interview which may be arranged anytime within the review period. Participants:

(Delegates)- Leadership Team, Coordinators, Staff identified by program



(SOP-Site-Level)- Program Officers, Site Supervisor, Teachers, FSW

2. Onsite Inspection (Classrooms, Outdoor Areas, Center or Facility): Onsite visits will take place during the identified review period. Agencies are highly encouraged to have an appointed staff/representative to be available for questions. QA staff and Program staff will also decide on the logistics of a daily debrief or mid-point check-in during the review period. IMPORTANT: Agency is responsible to inform or communicate with its own site staff (teaching and support staff) all pertinent details about the review prior to the site visits.

3. File Review/Documents Review (Onsite or Desk Audit): A portion of files will be randomly

selected and others will be selected for specific areas (e.g., IEPs, intensive health concerns, etc.). ChildPlus information and child file information will be used to assess compliance. QA staff will arrange with each program the logistics of child file review (e.g., location, access to files or electronic documents, etc.) IMPORTANT: To have a fair review, children’s files and pertinent documents should be complete and/or updated, orderly and readily accessible and available to the monitors onsite. Due to logistics and safety and supervision situation in the classrooms, monitors may not always have the opportunity to interview or ask teaching staff.

4. Staff and Parent Interviews: Interviews (scheduled or random onsite) of staff and parents will be part of the monitoring review process. There are performance indicators requiring responses from staff and parents to assess compliance.

5. Daily or Mid-Point Onsite or Program Debrief: A daily or mid-point check- in with staff will be part of the review process. During this short debrief, QA staff will:

Provide status or progress of the monitoring activities;

Ask site staff any issues or concern that may affect the review process

Notify site staff and appropriate supervisor of any item(s) that needs immediate correction of non-compliances/high-risk health and safety issues.

IMPORTANT: Any non-compliant item that is corrected satisfactorily onsite will still be marked Non-Compliant, but the indicator will have a completed CAP to denote that it has been corrected.

6. Release of Site-level monitoring results (Non-Compliant) with corresponding Corrective Action Plans (CAP) and timelines through ChildPlus:

Site-level monitoring results that are Non-Compliant with corresponding Corrective Action Plans (CAP) and timelines will be released to authorized agency program staff when it is cleared (typically within 5 business days of inspection/visit).



Release of site-level monitoring results can occur before a program-level report is generated.

Site-level monitoring results are provided in order for agency to correct quality and compliance issues at the class/center level as immediately as possible.

7. Program Summary Report Final Exit Interview:

Program summary report (trends report) will be available within 10 business days after all site visits have been completed.

Final Exit Interview will be arranged onsite or via conference call with agency staff.

8. ChildPlus Program Service Indicator Checklists: Indicator checklists will be utilized to monitor compliance in program service areas. Each checklist is comprised of performance indicators based on federal, state and local regulations, and countywide or agency-specific policies and procedures. The number of checklists used per review will vary on type of review (comprehensive or differential) and program option (center-based or home-based). While review is in progress, access to checklists on ChildPlus IMM is limited only to the QA Staff.

9. Data Quality and ChildPlus Desk Audit: ChildPlus is the required record-keeping system to be used by Delegates/Partners and SOP. Each agency is contractually obligated to have its ERSEA and Health Services recordkeeping updated by the 5th of each month. IMPORTANT: For 2017-2018, QA Monitors will review both children’s files and ChildPlus records to assess compliance in ERSEA and Health record-keeping. Starting in 2018-2019, monitoring of ERSEA and Health records will be by ChildPlus only. Programs need to put systems in place to ensure accurate and timely recordkeeping.

V. REPORTING PROCEDURES AND CORRECTIVE ACTION PLANS (CAP)

A. Monitoring Results, Reports and Corrective Action Plan (CAP) 1. Class/Center or Site Level and CAP: Monitoring results and CAP will be released to

designated staff via ChildPlus Internal Monitoring Module (IMM). There will be different due dates per CAP based on risk/priority level of non-compliance issues.

2. Program Level and CAP (Trends): Program-level Summary Report will be submitted to CFS Management and Delegate Administrators.

B. Closeout of CAP Action Items via ChildPlus IMM Designated agency staff are responsible for completing CAPs through the ChildPlus IMM system. Assigned staff will be provided access to their respective CAPs. Depending on the nature of the concern, submission of program documents to closeout CAP action items (e.g., photos, evidence of training, self-monitoring results) will be required.



C. Site visit verification or follow-up by QA staff Scheduled or random site visit by QA staff will be conducted to verify corrected non-compliance items.

VI. QA SUPPORT AND RESOURCES

A. Role of Delegate Liaison Each program is assigned a QA staff to serve as liaison between Grantee and Agency to help facilitate communication, and to provide support in addressing compliance and quality improvement efforts.

B. QA Solutions Toolkit/Website Resources (COMING SOON) Resources such as copies of required postings, web links, tips and guidance to support successful MCQI monitoring and quality improvement activities will be available on SETA Head Start website.

C. MCQI Review Evaluation/Survey Evaluation survey will be sent out to each agency at the completion of each MCQI review to solicit feedback and input for improvement.



APPENDICES

Appendix A – Unannounced Safety and Supervision Form and Procedure Page 1



Unannounced Safety and Supervision Form and Procedure Page 2



Appendix B – Child Plus Checklist Summary List Sorted by Program Service Area

Service Area Checklist # / Title Number

of Indicators

Checklist Type

F = File Review O = Observation I = Interview

Disabilities 007. Disability File Review 10 File Review (F)

Education 005. Education File Review 14 File Review (F)

Education 016. Classroom Observation 17 Observation (O)

ERSEA 001. ERSEA Interview 5 Interview (I)

ERSEA 002. ERSEA File Review 10 File Review (F)

ERSEA 011. ERSEA Services 6 File Review / Observation (F/O)

FCE 008. FPA Building & Follow Up 12 File Review (F)

FCE 013. Family Community Engagement – Facilities

8 Observation (O)

Health 003. Health Services File Review 29 File Review (F)

Home-Based 019. Home-Based Program 18 File Review/ Observation (F/O)

Home-Based 020. Socialization 37 Observation/Interview (O/I)

Home-Based 021. Services for Pregnant Women 24 File Review (F)

Mental Health 006. Mental Health File Review 8 File Review (F)

Nutrition 004. Nutrition Services File Review 13 File Review (F)

Nutrition 015. Meal Service 22 Observation (O)

PDM 022. PDM Staff Licensing Records 14 File Review (F)

PDM 023. PDM Program Training Records 4 File Review (F)

PDM 024. PDM Required Safety Training 10 File Review (F)

PDM 025. PDM Staff Qualifications and Competencies

12 File Review (F)

Safe Environments 009. Special Diets 4 File Review/ Observation (F/O)

Safe Environments 010. Medication 7 File Review/ Observation (F/O)

Safe Environments 012. Required Postings 21 Observation (O)

Safe Environments 014. Safe Environments 59 Observation (O)

Safe Environments 017. Outside Safe Environments 13 Observation (O)

Safe Environments 018. EHS Safe Environments 15 Observation (O)



Appendix C – MCQI Checklists Flow Chart by Program Type