richard p. barth & terry v. shaw october 23, 2014 cswe apm tampa, florida implications of cfsr 3...
TRANSCRIPT
Richard P. Barth & Terry V. ShawOctober 23, 2014
CSWE APMTampa, Florida
Implications of CFSR 3 for IVE Programs
Purpose of the Presentation
• To stimulate thinking and discussion about:– What is the meaning of CFSR 3?– Does it create new opportunities for SSW
University-Agency Partnerships?– Does it suggest changes in SW Education?
CFSR 3 Guiding Principles• States and the Federal Government are unified in a
commitment to safety, permanency, and well-being for the children served by the public child welfare system.
• The CFSRs have led to the identification of opportunities for:– 1. program improvement – 2. improvement in outcomes for children and families.
• Further improvements in the CFSR and in the related CQI processes are critical to the continued CWs advances– Scientifically valid– Feasible– Understandable– Meet All Federal Requirements
National Standards (Issues)Proposal
InputAggregate
administrative data on safety and
permanency
Applied Methodology Multi-level Model
Risk Adjusted
OutputSet National Standard
at national averagefor all measures
Points of Agreement
Proposal is reasonable
Areas for Discussion
Recommendation to endorse CB proposal; or
Compute State-specific rates and calculate a national
average rate
Recommendation to endorse CB proposal; or
Endorse with additional factors; or; Do not risk adjust
National Standards
CFSR Round 2 CFSR Round 3 Differences
Of all children in foster care during the reporting period, what percent were not victims of substantiated or indicated maltreatment by a foster parent or facility staff member?
Of all children in foster care during a 12-month period, what is the rate of victimization per day of foster care? (most likely reported in a rate per 100,000 days)
Child maltreatment while in care is calculated by directly linking data from NCANDS (numerator) with the AFCARS file (denominator) using CHILDID.
All instances of maltreatment will be included regardless of type of perpetrator (including parents).
Safety Outcome 1: Children are, first and foremost, protected from abuse and neglect.
Safety Performance Measure 1: Maltreatment in Foster Care
Re-Report of Maltreatment (Issues)
ProposalPoints of
AgreementAreas for Discussion
Of all the children who were victims of substantiated or indicated abuse or neglect during the first 6 months of the reporting period, what percent experienced another screened-in report with a 12-month period?
It is important to include a measure of repeat maltreatment. No support for the measure as proposed “…..getting the measure right should trump concerns over current data capacities.”
Recommendations for appropriate measure(s)? Recommendations around data availability, quality and testing proposed measures?
National Standards
CFSR Round 2 CFSR Round 3 Differences
Of all children who were victims of substantiated or indicated maltreatment allegation during the first 6 months of the reporting period, what percent were not victims of another substantiated or indicated maltreatment allegation within a 6-month period?
Of all children with a screened-in report of alleged maltreatment in a 12-month period, what percent had another screened-in report within 12 months of their initial report?
Lengthens the timeframes being considered (from 6 months to 12 months)
increases the scope to include all screened-in reports and not just Substantiated or Indicated.
Safety Outcome 1: Children are, first and foremost, protected from abuse and neglect.
Safety Performance Measure 2: Re-report of Maltreatment
National Standards
CFSR Round 2 CFSR Round 3 Differences
Composite 1.3: Of all children entering foster care for the first time in a 6-month period, what percent discharged to reunification (or live with relative) within 12 months of entering foster care or by the time they reached 18?
Of all children who enter foster care in a 12 month period, what percent discharged to permanency within 12 months of entering foster care?
Expanded the exits from reunification alone, to permanency (reunification, live with relative, adoption or guardianship). Expands base time from 6 months to 12 months and includes all entries, not solely first entries.
Permanency Outcome 1: Children have permanency and stability in their living situations.
Permanency Performance Measure 1: Permanency in 12 months for children entering foster care
National Standards
CFSR Round 2 CFSR Round 3 Differences
Composite 3.1: Of all children in foster care on the first day of a 12-month period who had been in foster care (in that episode) for 2 or more years, what percent discharged to permanency within 12 months of the first day or by the time they reached 18?
Of all children in foster care on the first day of a 12-month period who had been in foster care (in that episode) for 2 or more years, what percent discharged to permanency within 12 months of the first day?
No differences.
Permanency Outcome 1: Children have permanency and stability in their living situations.
Permanency Performance Measure 2: Permanency in 12 months for children in foster care for 2 years or more
National Standards
CFSR Round 2 CFSR Round 3 Differences
Composite 1.4: Of all children discharged from foster care to reunification or live with a relative in a 12- month period, what percent re-entered foster care in less than 12 months from the date of discharge?
Of all children who enter foster care in a 12- month period and discharged within 12 months to reunification, live with relative, or guardianship, what percent re-entered foster care within 12 months of their date of discharge?
The measure is now based on an entry cohort instead of an exit cohort.
The denominator has been expanded and now includes Guardianship as well as reunification, and live with relative.
Permanency Outcome 1: Children have permanency and stability in their living situations.
Permanency Performance Measure 3: Re-entry in 12 months
National Standards
CFSR Round 2 CFSR Round 3 Differences
Composite 4.1: Of all children served in foster care during the 12- month period, what percent had two or fewer placement settings?
Of all children who enter foster care in a 12- month period, what is the rate of placement moves per day of foster care? (most likely expressed as a rate per 1,000 days)
The measure is now based on an entry cohort instead of all children served and in care for a period.
All placement moves will be counted during a period rather than just identification of stable vs. not stable.
Permanency Outcome 1: Children have permanency and stability in their living situations.
Permanency Performance Measure 4: Placement stability
Risk Adjusted Analyses
Performance Goals and Thresholds (issues)
Proposal Points of Agreement
Areas for Discussion
InputState specific data on the most recent 3 years of performance
Applied MethodBootstrapping to set baselineEstimate grand mean and standard deviation
Out PutImprovement Factor Grand Mean/Grand Mean + 2SD Goal CalculationBaseline * Improvement FactorThreshold CalculationBaseline – 2SD
Applied MethodMin. & Max. Goals & ThresholdsMin. = Top of bottom 5th
Max. = Bottom of top 5thth Use Companion Measures
OutputMeasurement at 3 & 5 years for some measures and 4 & 5 years for one measurement
Bootstrapping is reasonable
Recommendation to incorporate trend consideration
Recommendation to adjust calculations to 1 SD
Recommendation to set at 5th and 45th State performance levelCompanion
Measures are reasonable
Measurement timeframes are
reasonable
Case Review Standards (issues)
Proposal Points of Agreement
Areas for Discussion
Case Review determines substantial conformity for 24 items that assess case-specific practices in both in-home and foster care cases.
A single standard is applied to all Case Review items.
Case Review is an important component of a CQI system. A single standard should be applied.
The Panel suggested that Case Review should not be used to determine Substantial Conformity. How would Case Review results be used to meet legislative objectives to measure State performance and improvement? What should the standard be?
Case Review Standards• In prior rounds onsite case reviews were conducted during
a one-week visit by external reviewers.• In round 3, qualifying states may conduct their own reviews
using a revised CFSR onsite review instrument. States have to meet three criteria in order to use their own case review process and be able to meet the following time lines.– Must use a sample period of 4/1 to 9/30 of year prior to review
year.– Conduct reviews from 4/1 to 9/30 of the year of review and
provide information to the Children’s Bureau– Report results no later than 11/15 of the year of review to the
Children’s Bureau
Case Review Standards
• Criterion 1 - The states operates an internal case review process at least annually that assesses statewide practice performance for the key child welfare areas using a uniform sampling process and methodology
• Criterion 2 - The state has a process in place for ensuring accurate and consistent case review ratings.
• Criterion 3 - The state uses the federal onsite review instrument and its instructions using the sample and method established above to collect data to be used for the initial determination of conformity.
Sampling (issues)
Proposal Points of Agreement
Areas for Discussion
Sample to allow for statistical inferences
Combined foster care and in-home case sample
Simple random sampling design
Statewide with options for geographic representation
Option to reduce sample size for ongoing program improvement
No support for a sampling approach that could eliminate children in rural areas No support for reducing sample size for ongoing program improvement
Should sample size be increased to allow for statistical inferences for foster care and in-home cases?
Is the proposed sample size feasible?
Is the proposed random sampling design feasible?
What are alternative sampling strategies?
Panel suggestion that targeted reviews could have a role in ongoing improvement monitoring.
Sampling
• Random selection; or a stratified selection representative of the states demographics and includes the largest metropolitan area.
• Minimum of 65 cases served during the sample period (40 foster care cases; 25 in-home cases, including Alternative Response);
• Out of home cases must be consistent with the listings reported to AFCARS during the period.
• In home cases (including AR) must have been opened for at least 45 consecutive days during the sampling period .
• The state must be able to consistently address document case elimination.
Standards for Systemic Measures (issues)
Proposal Points of Agreement
Areas for Discussion
The State’s CQI system would identify sources for quantifiable data for items TR1 – TR4 and FAP 2.
A single standard is applied to all systemic capacity items.
The prior method of assessing systemic measures was subjective.
The evidence connecting the systemic factors to family outcomes is limited.
What is the recommended standard and why?
90%75%
Should the standard be different for systemic capacity items assessed through case review?
Implications
1. The measures are simpler which will help in explaining what they mean to all the various audiences - this should make it clearer to develop strategies to influence outcomes.
2. Opportunities for SSWs to increase their role regarding development and implementation of CQI processes
3. Opportunities to work with state partners on the importance of understanding the trajectory of children into and through the service systems
4. Greater focus on CQI in social work education5. Critiquing CFSR 3 or Designing CFSR 4 could become a useful
activity for a research or CWS course