Penn State Clearinghouse for Military Family Readiness Perkins, D. F., Karre, J. K., DiNallo, J. M., Kyler, S. J., & Aronson, K. R.

The Pennsylvania State University, University Park, PA

Continuum of Evidence

Criteria Effective Promising Unclear Ineffective Significant Effect Rigorous statistical evidence

of a change in highly desired behavioral outcome that was considered significant, with no negative effects found

Rigorous statistical evidence of a change in highly desired behavioral outcome that was considered significant, with no negative effects found

Effects are unclear due to mixed results or limited evaluation design

An appropriate evaluation has failed to demonstrate a significant effect, or has negative effects

Sustained Effect Effect lasting ≥ 2 years from the beginning of the program, or ≥ 1 year from program completion

Effect(s) lasting or ≥ 1 year from beginning of the program, or ≥ 6 months from program completion; Noted considerations may be given for programs that have not had sufficient time to demonstrate long-term effects.

Sustainability not assessed or established

Program effects not sustained

Successful External

Replication

Program found effective in at least one other randomized controlled trial conducted by an implementation team that was independent of the program developer

No evidence of external replication, or limited replication criteria (i.e., lacking significant/sustained effect, inadequate study design)

No evidence of external replication

No evidence of successful external replication

Study Design Randomized controlled design Uses at least a quasi-experimental design

Uses at least a quasi-experimental or pre-post-test design, or purely descriptive

Experimental or quasi-experimental design

Additional Criteria Regarding Study

Design

Most (i.e., ≥ 50% [4/8]) of the additional criteria have been addressed

Some (i.e., 25% to 49% [2/8]) of the additional criteria have been addressed

Few to none (i.e., < 24% [<2/8]) of the additional criteria have been addressed

Most (i.e., ≥ 50% [4/8]) of the additional criteria have been addressed

What is the Clearinghouse? Sample Fact Sheet

Dissemination & Implementation Continuum Methodology

The Penn State Clearinghouse for Military Family Readiness (Clearinghouse) is a distribution and implementation infrastructure for professionals (i.e., practitioners and researchers) supporting military families.

The Clearinghouse is designed to be an interactive resource involving the selection, dissemination, and evaluation of evidence-based programs and practices for professionals. Utilizing a dynamic web platform as the backdrop, the Clearinghouse provides a proactive connection with professionals through quality technical assistance and problem-solving to help professionals make informed decisions regarding how best to serve military families.

The Clearinghouse began forming its infrastructure by developing the Continuum of Evidence to be employed as the criterion that guides the population of its website with evidence-based programs and practices. To continue the process of forming a distribution and implementation infrastructure, the Clearinghouse: •  Actively engages military family support professionals with multiple capacity-building activities

designed to foster learning and promote efficient implementation of effective evidence-based programs;

•  Identifies evidence-informed practices; •  Provides ongoing and proactive technical assistance about implementation; •  Synthesizes existing and emerging research from a variety of sources on

various topics; •  Develops user-friendly practical guides, research reports, and briefs; •  Reviews and improves homegrown military programs for children and

families based on established standards of evidence; and •  Is easily accessible to professionals on military bases around the world, as well as to those who

work in communities where Active Duty, National Guard, and Reserve military families live, work, and attend school.

The Continuum of Evidence provides a well-defined and useable framework for the Clearinghouse to identify evidence-based programs and practices, and to provide proactive implementation support. The Clearinghouse’s Continuum of Evidence is designed to find, evaluate, and place existing programs/practices on a continuum based on level of effectiveness as supported by empirical evidence. The development of the Continuum was a one-year process involving two major steps. First, existing evaluation systems with a rigorous level of criteria for evidence were carefully examined1. The Office of Justice Programs (OJP) What Works Repository was found to provide the most comprehensive and rigorous criteria, and was therefore utilized as the primary guide for the development of the final Continuum of Evidence. Second, the Clearinghouse engaged in an iterative process with staff for the Office of the Deputy Under Secretary of Defense for Military Community and Family Policy, and in an editing process related to usability for professionals. The final Continuum was approved by our Department of Defense Partners.

1The existing evaluation systems that were examined included (1) OJP What Works Repository (2004), (2) Blueprints for Violence Prevention, (3) National Registry of Evidence-based Programs and Practices, (4) What Works Clearinghouse, (5) Promising Practices Network on Children, Families, and Communities, (6) Model Program Rating System, (7) Guide to Community Preventive Services, and (8) The Society for Prevention Research.  This material is based upon work supported by the National Institute of Food and Agriculture, U.S. Department of Agriculture, and the Office of Family Policy, Children, and Youth, U.S. Department of Defense under Award No. 2010-48709-21867 developed in collaboration with The Pennsylvania State University.