systems for actionsystemsforaction.org/sites/default/files/resource_files/rwjf s4a... · 2....

Systems for ActionSystems and Services Research to Build a Culture of Health

Research Agenda DevelopmentDelphi Panel Stage 3 Results

May 2015

S4A_Delphi_0915

www.systemsforaction.org

2 | Systems for Action National Program OfficeDelphi Panel Stage 3 Results

Systems for Action Research Agenda Development

Delphi Panel Stage 3 Results

This report was prepared for discussions of the Systems for Action Technical Advisory Committee, and includes the results of a Delphi survey process used to identify priority topics to be included in the Systems for Action Research Agenda.

Table of Contents

RESEARCH AGENDA DEVELOPMENT PROCESS........................................................................... 3

NON-RANKED RESULTS: TOPICS ORDERED BY THEIR POSITION ON THE SURVEY ................... 5

RANKED RESULTS: TOPICS ORDERED FROM LOW (MOST IMPORTANT) TO HIGH (LEAST

IMPORTANT) BASED ON THE STANDARDIZED MEAN SCORE ................................................... 9

Center for Public Health Systems and Services Research University of Kentucky College of Public Health May 5, 2015


RESEARCH AGENDA DEVELOPMENT PROCESS The Robert Wood Johnson Foundation (RWJF) appointed a Technical Advisory Committee in February 2015 to develop a research agenda for a new Systems for Action (S4A) national research program. The Committee included ten representatives with relevant expertise in areas that include medical care, nursing, health policy and management, economics, community and stakeholder engagement, social and organizational systems, and health equity. Because the committee members were geographically dispersed across the United States, research agenda development was completed from March through July using a variety of deliberation mechanisms, including electronic communications, two virtual meetings, and one in-person meeting. Committee members recommended engaging additional stakeholders to enrich discussion of potential topics for the S4A research agenda, and members were subsequently invited to nominate additional stakeholder representatives to participate in agenda setting activities. Three additional stakeholders joined the agenda and priority-setting process representing diverse perspectives, including two representatives having experience in community development and engagement with underserved racial and ethnic groups in health research, and one representative having experience with stakeholder engagement of health care professionals and interest groups in quality measurement and reporting activities. As background for identifying research priorities, committee members were provided an overview of the Robert Wood Johnson Foundation (RWJF) Culture of Health Action Framework and a synthesis of public health services and systems research evidence from recent studies. A three-stage Delphi survey process was used to identify and prioritize S4A research topics with participation by the committee members and stakeholder representatives (n=13), RWJF representatives (n=2), and key project staff (n=3). A secure electronic survey tool was used throughout the process. In the first stage, potential topics and research areas were solicited by asking each person to submit between three and twelve candidate research topics, considering these four criteria:

1. The potential for research on the topic area to generate knowledge that leads to significant improvements in health status and health equity through relevant components of RWJF’s Culture of Health Action Framework, i.e., health as a shared value, cross-sector collaboration, healthy and equitable communities, and integrating health and health care systems;

2. Relevance to the S4A general theme of aligning and integrating services and delivery and financing systems that impact population health, including public health, medical care, and social and community services;

3. The potential for research on the topic area to generate new knowledge and evidence that does not already exist; and

4. The potential for research on the topic area to complement and be synergistic with—and not duplicative of—research supported by other funders and funding mechanisms.

Nominated topics were solicited from committee members and stakeholders using the secure electronic survey tool, and a total of 55 topics were received during the first stage solicitation. In the second stage, committee members and stakeholders were asked to rate each of the 55 nominated topics on a 10-point scale, ranging from “Very Important” to “Not Important,” considering the same criteria listed above. Respondents also were invited to nominate up to five additional topics for consideration. During this rating process, nominated topics were not edited, combined, or divided except in obvious cases of duplication; however some overlap in topic areas was addressed later in the research agenda-setting process. After the second stage ratings were completed, rating results were disseminated back to the respondents, including individual rater results as well as statistics for central tendency, range, coefficient of variation, and other measures of agreement in ratings.

http://www.systemsforaction.org/projects/research-agenda/reports/research-summaries-systems-and-services-research-relevant-systems-action-national-research-program


In the third stage, the panel reviewed the group and individual ratings of the first 55 topics and were prompted to confirm or change their ratings for each topic after having reviewed the ratings of other panelists. In addition, panelists rated the importance of 11 new topics recommended in the second stage survey, using the same 10-point scale as above. With the third stage ratings completed, standardized rating scores and measures of agreement were calculated for each of the 66 topics. Topics were ranked from most important to least important, based on the standardized mean score. Committee members and stakeholders received the rank-ordered topic list with detailed results on ratings, which are included in this report. Panelists were provided with research evidence summaries completed in 11 broad areas related to the nominated topics, including research on delivery systems for social services, community development, and poverty reduction. Summaries, while not comprehensive evidence reviews, were designed to stimulate further thinking and dialogue about S4A research priorities. These summaries are available in a separate report. An in-person meeting of committee members and stakeholders was conducted to refine, consolidate, de-duplicate, and prioritize the list of 66 research topics. Committee members who were not able to attend the meeting were interviewed individually to gather their opinions into the process. The ten topics with the highest mean standardized importance ratings identified in the third stage Delphi survey were provided as a starting point for the convergence discussion, with additional items grouped accordingly. Some topics were deemed more fitting as guiding principles, methodological approaches, or dissemination and translation recommendations. After the in-person meeting, written descriptions of priority S4A research agenda items were developed, reviewed and refined through three waves of written comments and telephone conference calls held with committee members and stakeholder representatives.

http://www.systemsforaction.org/projects/research-agenda/reports/research-summaries-systems-and-services-research-relevant-systems-action-national-research-program


NON-RANKED RESULTS: TOPICS ORDERED BY THEIR POSITION ON THE SURVEY

RWJF SYSTEMS FOR ACTION RESEARCH DELPHI PANEL RESULTS | Stage 3 Items Ordered by Position on Delphi Survey (Lowest Score = Most Important)

Item Mean Median St. Dev Mode Min Max Mean CV ICC %Below Rank A B C D E F G H I J K L M N O P1. Examine the effectiveness of integrating a core set of community-based public health indicators intoelectronic health records in developing transitions of care and management strategies to improve outcomes and reduce the burden of chronic disease.

2.63 2.00 1.71 1.00 1.00 6.00 -0.56 -2.00 0.23 69% 6 1 1 2 1 5 1 3 1 2 2 2 2 5 3 6 5

2. Evaluate a core set of public health system/population health measures to drive high quality public healthperformance, for their alignment with health care performance, and feasibility, usability, and importance in public health systems and services research

3.69 4.00 1.40 5.00 1.00 5.00 0.15 5.91 0.14 44% 45 5 3 4 1 4 1 4 3 5 2 3 5 5 4 5 5

3. Research on approaches to system coordination and in particular electronic record environments thatfacilitate, rather than impede getting data out of systems, sharing data or using and analyzing data for system and/or community quality improvement efforts.

3.63 4.00 1.63 4.00 1.00 6.00 0.10 9.71 0.16 44% 41 4 1 3 4 2 3 2 1 4 4 6 3 5 4 6 6

4. Research to more systematically make available multiple level indicators for public health and healthcare practitioners (e.g., zip-code specific risk factors for major diseases, individual behavioral practices and healthcare access), to create both an infrastructure that promotes rather than inhibits using data from electronic systems; and makes available data to practitioners that is multiple-layered and contextualized. This will also require retraining of clinicians to use such layered data to inform public health and clinical practice.

3.75 4.00 1.34 4.00 2.00 6.00 0.22 3.85 0.13 44% 49 5 3 2 4 4 3 2 2 4 4 4 2 5 4 6 6

5. Financing and organizational models that promote collaboration across public health, healthcare, communityand social infrastructures (e.g., schools, social and community services, businesses) to improve important health issues, particularly addressing disparities in health and risk factors for health, across neighborhoods. Research is needed to identify incentives, pilot and align incentives across diverse community sectors, and determine effects on specific key health indicators. Health indicators should be those prioritized by community stakeholders including the most vulnerable, and mechanisms should include evidence-based and community-defined evidence especially for drivers that have less been the subject of at-scale intervention research (e.g., discrimination, marginalization).

1.69 1.00 1.14 1.00 1.00 5.00 -1.13 -0.59 0.06 94% 1 3 1 5 1 1 1 1 1 1 1 1 1 3 2 2 2

6. Determine whether and how inter-sectoral collaboration between the health care system and social services (through effective referral systems for example) improve outcomes of care.

1.94 2.00 0.68 2.00 1.00 3.00 -0.94 -0.62 0.03 94% 4 2 1 1 2 2 2 2 3 2 1 3 1 3 2 2 2

7. Compare the effectiveness of setting population health priorities using community health planning strategies developed in a health system alone versus a health system engaged with public health and community organizations.

3.50 3.50 1.37 4.00 1.00 6.00 -0.04 -14.44 0.02 44% 33 4 2 5 2 4 4 1 2 4 3 5 6 3 3 5 3

8. Developing and testing interventions and methods to optimize the impact of organizational partnerships onimproving processes (accreditation aligned outcomes) in local public health systems.

3.63 3.50 1.89 3.00 1.00 7.00 0.03 37.95 0.19 44% 38 5 1 3 2 5 1 3 3 4 4 4 7 1 3 7 5

9. Are health outcomes better in jurisdictions in which health departments forge stronger and more numerous multi-sectoral partnerships?

3.53 3.00 2.03 2.00 1.00 8.00 -0.08 -18.95 0.34 63% 29 2 0 2 1 8 2 2 4 3 3 4 2 6 3 7 4

10. Research development among community stakeholders in collaboration with public health and healthcare practitioners, to have a community infrastructure to identify key needs, promote understanding of research among stakeholders, develop relationships with system and services researchers, and have resources to incentivize/encourage researchers to work with community stakeholders in addressing local and national culture of health, with strong inclusion of more vulnerable stakeholders in leading the community infrastructure for research (e.g., community centers of excellence for health research).

2.88 2.50 1.54 2.00 1.00 6.00 -0.35 -2.37 0.12 69% 18 3 2 4 3 1 4 2 3 5 1 5 1 2 2 6 2

11. Compare the effectiveness of alternative policy and environmental change strategies to achieve long-term, sustained increases in physical activity. Also, look at socioeconomic and racial/ethnic differences.

3.31 3.50 1.85 4.00 1.00 8.00 -0.15 -5.76 0.13 50% 25 5 4 3 4 4 2 1 1 5 2 3 8 1 4 2 4

12. Research on the effectiveness of food and nutrition assistance (SNAP, WIC, school breakfast and lunch) andeducation (e.g. SNAP-Ed) on health outcomes. Study populations would include eligible participants and eligible nonparticipants, both cross-sectionally and longitudinally.

4.38 4.50 1.71 5.00 2.00 8.00 0.52 1.54 0.11 19% 60 2 4 5 5 4 6 3 3 5 2 6 8 4 5 2 6

13. Research to determine communication, engagement and motivational strategies to increase community andpublic awareness of equity, including dissemination of findings of the impact of community health indicators on the health of all and models for improving health of all by improving health of the most vulnerable. Within this research, there should be a focus on a comprehensive model of health spanning physical, emotional/psychological, and social well-being as well as on environmental and structural community drivers of health.

3.31 3.00 1.85 3.00 1.00 7.00 -0.16 -6.76 0.22 69% 24 3 5 6 3 1 2 5 1 3 5 4 3 1 2 7 2

14. Compare the effectiveness of alternative policy and environmental change strategies to achieve long-term, sustained improvements in nutrition. Also, look at socioeconomic and racial/ethnic differences.

2.88 2.50 1.96 1.00 1.00 8.00 -0.45 -2.09 0.16 69% 11 2 4 3 1 4 1 1 2 5 2 3 8 1 5 1 3

15. What is the impact of sharing information on small area variations in health with local communities on their understanding of (1) health determinants or (2) the policies and actions that may be most effective to improve health?

3.94 4.00 1.18 4.00 2.00 6.00 0.31 2.63 0.11 44% 52 4 5 4 3 2 4 6 2 5 4 3 4 4 4 3 6

16. Comparing the reach and effectiveness of public health services and/or policies to reduce and/or eliminate racial and ethnic disparities on health outcomes with the widest Black-white disparities, Latino-white disparities, and/or other subgroup disparities: http://aje.oxfordjournals.org/content/166/1/97.full.pdf+html

1.75 1.50 0.93 1.00 1.00 4.00 -1.10 -0.48 0.02 94% 3 1 1 3 1 1 1 2 1 3 1 2 2 1 4 2 2

17. What types of small area variations in health (and health inequalities across small areas) can be effectivelycharacterized using routinely and newly available data (including 'BIG data?) and state-of-the art methods?

4.25 4.00 1.29 5.00 2.00 7.00 0.49 1.43 0.07 31% 59 3 4 7 3 6 5 2 3 5 3 4 5 5 4 4 5

18. Advancing the use and examination of the utility of agent-based modeling, social network analysis, andother system science methods in improving population health. Examining the effectiveness and use of these methods and results to facilitate strategic and operational changes in public health organization and delivery through local public health and community stakeholder engagement.

4.19 4.00 1.38 5.00 2.00 7.00 0.45 1.79 0.11 25% 56 4 2 3 4 4 5 4 3 5 5 5 7 5 3 6 2

19. Can we measure differences in resiliency between communities with greater integration of health and social services compared to those with lesser integration?

3.75 4.00 1.77 4.00 1.00 8.00 0.23 4.72 0.21 38% 50 3 4 6 4 8 1 3 4 4 1 5 2 5 3 3 4

20. Can we measure differences in resiliency between communities with greater health equity compared tothose with lesser health equity?

4.00 4.00 1.75 4.00 1.00 7.00 0.32 2.85 0.15 25% 53 6 4 6 4 4 1 2 4 4 1 5 7 5 2 4 5

21. Research examining and learning from positive deviance in improving local population health and reducing disparities in local population health outcomes, including how local stakeholders engage communities and use data to drive improvement.

2.81 3.00 1.33 2.00 1.00 5.00 -0.43 -2.00 0.13 75% 12 4 2 4 4 1 3 1 2 2 5 3 3 1 3 5 2

22. Research on the added value of diverse technology resources (e.g., social media and internetcommunication and intervention) to increase the efficiency of change in culture of health-related values of communities, systems and providers; to facilitate access to information for planners to create healthier environments; and to reinforce culture of health values over time and provide updates on mechanisms (whether individual or family coping or community improvement strategies), transparently available. Strategies should be available to increase the transparency concerning risk factors that apply to individuals and communities and strategies available and particularly if evidence-based, suitable for diverse languages, educational levels, from leaders to community members and patients.

4.88 5.00 1.67 5.00 1.00 8.00 0.90 0.92 0.12 19% 65 6 6 8 5 4 6 1 3 5 5 7 5 4 5 5 3

23. Research on the long-term health value of investments in healthier families and children, building resiliencyand coping, and resiliency to crises for individuals, families and communities, including in response to diverse disasters and environmental crises to community violence and major family traumatic events. That is, research that shows the value of particular and integrated strategies to invest in healthier families and children, both for short-term benefit and long-term value, either through natural experiments or long-term follow-up of community change efforts and especially their impact on children.

3.44 3.00 1.15 3.00 2.00 6.00 -0.04 -16.09 0.06 63% 31 4 3 5 4 3 2 4 2 3 5 3 6 3 3 3 2

24. Research that includes in the building of healthier communities, a focus on prevention, early interventionfor social and structural risk factors for emotional well-being and long-term investment in development, particularly of children and families. For example, work on comprehensive public health models to address impact of trauma and prevent early consequences of child exposure to trauma or create safer environments to reduce trauma exposure, e.g., safe spaces in terms of violence, trauma, domestic violence -- areas that from a social determinants of health or prevention perspective, tend to be de-emphasized over many years as part of health promotion and interventions for social determinants.

3.44 4.00 1.79 4.00 1.00 7.00 -0.09 -9.42 0.14 44% 28 4 5 5 4 1 2 5 1 3 2 5 7 4 2 4 1

Raw Scores Individual Reviewer RatingsStandardized Scores

6 | Systems for Action National Program Office--Delphi Panel Stage 3 Results


Item Mean Median St. Dev Mode Min Max Mean CV ICC %Below Rank A B C D E F G H I J K L M N O PRaw Scores Individual Reviewer RatingsStandardized Scores

25. Research on community engagement strategies, from relationship building to technology (as above) toincentives (above) and use of creative arts, empathy, to increase connection of people, increase motivation and commitment to equity in health outcomes and community building efforts. In addition, research may be needed on how the research process itself can be enhanced to have more community engagement as part of the design and conduct of research to build a culture of health.

3.56 4.00 1.93 4.00 1.00 8.00 0.00 -743.25 0.21 44% 37 3 2 4 5 1 4 4 3 4 6 5 8 1 2 4 1

26. Rigorous research and evaluation is needed on how leaders are created and supported, and the progressionof their leadership, in culture of health pathways, across a range of leaders (community, patients, public health, providers, administrators, policymakers), which may inform future and current human capital programs.

4.69 4.50 2.15 5.00 2.00 9.00 0.70 1.36 0.17 25% 63 4 4 5 5 2 5 5 4 4 2 9 9 3 7 5 2

27. Paraprofessional training to advance health via communities where they live and work. For example, community health workers are provided technical training in health topics that they promote to community members. Potential research: standardization of technical training on a particular health topic or general of training hub for paraprofessionals.

3.69 3.50 1.74 3.00 1.00 8.00 0.09 11.09 0.17 50% 40 4 4 3 5 3 3 5 1 4 5 3 8 1 5 3 2

28. In communities that have adopted Health in All Policy approaches can we demonstrate improved health? 4.25 4.00 2.08 4.00 1.00 8.00 0.48 2.44 0.25 44% 58 4 8 3 5 6 3 2 4 2 1 7 4 4 6 7 2

29. What is the impact of community development policies on health? Under what conditions are these policiesmost effective at improving health? What are the most important components of these policies?

2.94 3.00 1.44 3.00 1.00 7.00 -0.31 -2.89 0.14 69% 21 2 7 3 3 3 4 1 3 2 2 3 1 4 4 2 3

30. Compare the effectiveness of changes to the built environment (e.g. city code, complete streets, urbanplanning) on health. Develop models for impact on health and economic impact.

2.88 2.00 1.82 2.00 1.00 8.00 -0.34 -3.35 0.24 69% 19 5 3 8 2 4 2 1 4 2 2 1 3 4 1 2 2

31. To what extent can community interventions or policy interventions not directly related to health enhance the effectiveness of traditional individual-level interventions (including health education as well as medical treatments)?

3.38 4.00 1.41 4.00 1.00 5.00 -0.01 -139.80 0.10 44% 35 3 5 5 4 2 1 1 4 4 2 5 2 4 5 3 4

32. Is there a willingness at the community level (through taxes, programs, etc.) to support cross sectoral effortsto improve health and wellbeing?

4.81 5.00 2.14 6.00 2.00 9.00 0.80 1.14 0.15 31% 64 5 6 6 5 2 6 2 3 6 2 9 8 3 6 5 3

33. Environmental health as it is related to environmental justice issues. This topic would incorporate issues ofinequity based on location of pollution and contamination as well as income, education, and ethnicity. This links to 'creating healthier, more equitable communities' mandates. Potential for research: 1) ethnographic study of the culture of pollution and 2) community based-research of environmental issues and links to community health.

3.88 4.00 2.06 2.00 1.00 8.00 0.20 6.10 0.26 44% 47 2 6 5 4 4 3 7 3 5 5 4 8 1 2 2 1

34. Role of multiple technical sectors, such as Agency for Toxic Substances and Disease Registry and the Environmental Protection Agency, contributing to the culture of health and environmental health of local communities.

4.06 4.00 1.12 4.00 1.00 5.00 0.36 2.29 0.11 25% 55 4 4 4 4 5 4 4 4 5 5 5 5 1 2 4 5

35. To what extent does the consideration of social information in the clinical encounter (such as through the inclusion of information on social determinants in the electronic health record) improve the outcomes of clinical care.

3.20 3.00 1.42 4.00 1.00 7.00 -0.28 -3.87 0.21 69% 22 4 7 2 0 3 3 4 4 2 2 4 1 3 4 3 2

36. Compare the effectiveness of alternative redesign strategies--using decision support capabilities, electronic health records, and personal health records--for increasing health professionals' compliance with evidence-based guidelines and patients' adherence to guideline-based regimens for chronic disease care.

3.75 4.00 1.98 4.00 1.00 9.00 0.13 7.37 0.17 50% 43 5 4 3 3 5 6 1 1 4 2 9 4 3 4 4 2

37. Can we demonstrate improved health outcomes (short and long term) when there is closer integration inthe health delivery system between health care and public health? Between mental health and health care? Between mental health and public health?

1.75 2.00 0.68 2.00 1.00 3.00 -1.12 -0.43 0.00 100% 2 2 2 2 1 2 1 1 1 1 1 2 2 3 2 3 2

38. Where population health is a shared responsibility between public health and health care, are healthoutcomes better?

2.50 2.00 1.32 2.00 1.00 5.00 -0.66 -1.09 0.08 81% 5 2 5 2 1 2 1 1 3 2 1 3 4 3 3 5 2

39. Research on the intergenerational transmission of poverty on health outcomes, comparing residents inpersistently poor communities versus residents in non-poor communities. Related will be whether social services (access and use of both health and cash/in-kind transfers) mediate the transmission of poverty into poor health.

3.81 3.50 2.20 4.00 1.00 9.00 0.15 7.40 0.23 44% 46 1 6 4 4 5 3 3 3 5 2 1 9 4 2 2 7

40. Research on the role of growing up in communities with high income inequality versus low income inequality on health outcomes, both contemporaneously and over the life course. Are certain public policies more effective than others in ameliorating the role of poverty and inequality on poor health of infants and adolescents?

3.56 3.00 2.28 1.00 1.00 9.00 -0.05 -21.85 0.20 50% 30 1 6 5 3 3 4 1 3 4 1 2 9 2 4 2 7

41. Comparing the effectiveness of centralized, public health delivered services vs. decentralized, practice-baseddelivery of services on the reach and effectiveness in improving population health outcomes. This line of research would be comparing delivery in different settings / accountability structures.

4.50 4.00 1.75 4.00 1.00 8.00 0.58 1.67 0.17 31% 61 3 1 4 4 4 7 6 4 4 4 6 8 3 3 6 5

42. Development and testing of policies and interventions to improve workforce satisfaction among state andlocal public health workers (recent analyses of PHWINS data highlight major opportunities for improvement).

5.19 5.00 2.34 5.00 1.00 9.00 1.01 1.12 0.23 25% 66 5 3 7 5 4 3 5 3 6 5 8 9 1 7 9 3

43. Linguistic and culturally appropriate approaches to translating health outcomes. Also, can also enveloptraditional practices that are customary in a culture. This would benefit the initiative's goal of improving health for marginalized populations. Potential for research on emerging areas in culturally based health practices.

4.69 5.00 2.06 5.00 1.00 8.00 0.69 1.44 0.18 19% 62 5 6 7 5 5 6 5 1 5 3 5 8 2 4 7 1

44. Identify recommended levels (per capita) of local and state investment in evidence-based public healthinterventions, and potential health and economic impacts.

3.13 3.00 1.96 3.00 1.00 9.00 -0.31 -2.36 0.08 75% 20 3 3 5 3 3 2 1 1 3 3 2 9 3 3 5 1

45. Natural experiments of the impact of changes in local public health delivery system and/or financing onpopulation health.

2.69 3.00 1.20 3.00 1.00 5.00 -0.54 -0.92 0.01 88% 7 3 2 4 3 2 2 1 1 3 1 4 3 2 3 5 4

46. In communities with greater social investment is there better health status per dollar of healthexpenditures?

2.81 3.00 1.05 3.00 1.00 4.00 -0.42 -1.48 0.05 75% 13 3 4 4 4 3 2 4 1 3 1 2 3 3 2 4 2

47. Compare the effectiveness and cost-effectiveness of evidence-based public health interventions to clinical interventions addressing the same issue.

3.63 3.50 1.50 4.00 1.00 7.00 0.03 19.16 0.06 50% 39 4 2 3 4 3 3 1 2 3 4 7 6 3 4 4 5

48. Compare the effectiveness of combinations of evidence-based public health programs rather than isolatedinterventions to identify potential synergistic effects.

3.60 3.00 1.40 3.00 1.00 6.00 -0.09 -10.34 0.17 56% 27 5 3 3 3 4 2 1 2 5 5 5 6 3 0 3 4

49. Research that clarifies the added value of investments that communities make in intervening to improve health through different avenues, ranging from improving healthcare access and quality to improving education, poverty, addressing violence, etc., that is not only general (e.g., from the literature) but can be estimated and tailored for specific neighborhoods based on multi-level data on actual risk factors and health condition prevalence and structural characteristics of the zip code; both a model and mechanism to both estimate risk locally and estimate potential value of investments in action at environmental structural, system levels across diverse sectors.

2.94 3.00 0.85 2.00 2.00 4.00 -0.35 -1.42 0.01 81% 17 4 3 4 3 2 2 2 2 3 4 4 3 2 2 4 3

50. Compare the population-level health and economic impact of alternative CMS-funded State InnovationModels (SIMs) that attempt to align payment systems across multiple health care payers and incorporate incentives and metrics for improving health status on a population-wide basis. (Note: CMS is not funding rigorous research on these SIM projects so there is a need for research).

3.20 3.00 1.78 3.00 1.00 6.00 -0.38 -2.84 0.21 63% 15 2 5 6 1 0 5 2 1 2 3 5 3 3 3 1 6

51. Compare the health and economic impact of alternative ways of structuring and delivering housing assistance programs and policies through HUD, state and local housing authorities, and community redevelopment/reinvestment programs.

3.13 3.00 1.15 3.00 1.00 6.00 -0.24 -2.81 0.06 69% 23 2 4 6 3 2 4 4 3 3 1 3 4 3 3 2 3

52. How do federal, state, and local transportation projects, policies, and financing mechanisms impactpopulation health through pathways that include active commuting, injuries, and air quality? What are the net impacts, interactive and synergistic effects, and distributional effects on population subgroups stratified by SES, age, and geography (e.g. rural/suburban/urban)?

3.69 3.50 1.82 4.00 1.00 8.00 0.13 7.95 0.18 56% 42 4 4 8 4 3 3 1 4 3 2 3 5 4 2 2 7

53. Compare the population-level health and economic impact of alternative community benefit activities undertaken by tax-exempt hospitals as required by the enhanced IRS 990 regulations. How do the incidence, magnitude, and distribution of these impacts vary based on community benefit investment levels, based on institutional and community factors (including connectivity between hospitals and community organizations), and based on the processes used for community health needs assessment and priority-setting?

3.88 4.00 2.00 4.00 1.00 8.00 0.24 4.83 0.23 50% 51 4 3 4 2 6 2 5 4 5 1 3 7 4 3 1 8



Item Mean Median St. Dev Mode Min Max Mean CV ICC %Below Rank A B C D E F G H I J K L M N O PRaw Scores Individual Reviewer RatingsStandardized Scores

54. How do community-level availability and reach of nonmedical public health programs (e.g. tobacco, obesity, injury, infectious disease control, food safety) and social programs (e.g. nutrition and food assistance, housing, income and child support) affect health insurance coverage and costs via Medicaid, Medicare, and private insurance (employer-based or exchange)? Are there public and private health insurance cost offsets attributable to nonmedical public health and social services, and if so how do offsets vary based on the extensiveness and intensiveness of available nonmedical services/programs?

2.81 2.00 1.91 2.00 1.00 8.00 -0.49 -1.90 0.16 75% 10 2 2 3 2 4 2 1 4 3 1 2 8 2 2 1 6

55. How do the prevalence and intensiveness of employer-supported and worksite-based health promotion andwellness programs affect community-level health and economic outcomes? Are there community-level spill-overs and positive externalities associated with employer-supported activities? Are there community-level network and peer effects related to the adoption, spread, and effectiveness of these activities among employers, employees, families, and communities?

3.44 3.50 1.41 4.00 1.00 6.00 -0.01 -53.58 0.10 69% 34 5 4 4 3 3 2 1 4 5 2 4 5 2 3 2 6

56. What is the community economic impact resulted from the increased awareness of healthy life styles andbehavioral modification for the prevention of chronic illnesses and public health interventions (reduction of risk behaviors )?

3.80 3.00 1.82 3.00 1.00 7.00 0.00 -273.56 0.18 56% 36 5 5 0 3 3 2 1 2 3 3 7 7 3 6 4 3

57. What programs are actually available and perceived as available in under-resourced areas and how does that compare to the perception of policymakers and healthcare and public stakeholders? How far have things gone (e.g., denial of programs have put children at risk for long-term health loss). How can 'public' programs actually become meaningfully available to impact under-resourced communities and what public commitment would that take?

3.93 3.00 2.09 3.00 1.00 8.00 0.13 9.83 0.30 50% 44 2 6 0 5 1 3 3 6 5 1 6 8 3 5 3 2

58. How can social media create, amplify, or replace community connections toward improved health?4.47 5.00 1.81 5.00 1.00 7.00 0.47 2.56 0.26 19% 57 5 4 0 5 4 5 1 3 4 5 7 6 4 7 6 1

59. How can we address persistent discrimination at institutional, provider and individual levels to advance outcomes for under-resourced populations? What is the role of discrimination in undermining a culture of health? That is, how much is racism and discrimination impacting health outcomes and delivery of healthcare and public health services?

3.00 3.00 1.36 3.00 1.00 5.00 -0.36 -2.69 0.17 63% 16 3 4 0 3 1 2 3 4 5 1 4 3 2 5 4 1

60. Understanding opportunity and limits of the health care delivery system role in improving communityhealth. I thinking about healthcare providers working with providers of other social services to improve health. Understanding the benefits and also anticipating the unintended consequences.

3.60 4.00 1.40 2.00 2.00 6.00 -0.04 -23.66 0.17 50% 32 5 6 0 5 4 3 2 2 4 2 5 3 2 4 5 2

61. Maximizing the value of the full continuum of care from self care, informal care, and various level of care provided by healthcare providers. Understanding how patient, caregivers, providers and payers can work together to get the most from our resources we are devoting to improving healthcare outcomes.

3.43 3.00 2.03 4.00 1.00 8.00 -0.42 -2.57 0.21 63% 14 0 4 0 3 2 4 2 1 4 1 7 8 3 4 3 2

62. Achieving health equity and eliminating healthcare disparities. Identifying best practices and strategies thatimprove health outcomes for vulnerable populations. In particular, studying care delivery for racial and ethnic minorities, low income persons, persons with behavioral health problems, and chronically ill persons.

2.87 3.00 0.99 3.00 1.00 5.00 -0.50 -1.68 0.12 88% 8 2 3 0 2 3 3 5 1 4 2 4 2 3 3 3 3

63. Financial and other kinds of incentives in healthcare and their impact on access, costs and quality. Exploring and apply lessons learned from the role of behavioral economics and other models of human behavior from sociology and psychology to change the culture of health.

3.00 3.00 1.11 3.00 1.00 5.00 -0.50 -1.80 0.14 75% 9 3 2 0 2 3 4 2 2 4 1 4 3 4 5 3 0

64. The health care safety net in the age of Medicaid expansion and QHPs. How does the safety net evolve inera where most persons have coverage? What old problems persist? What new problem arise. What are the successes?

3.67 4.00 2.02 5.00 1.00 7.00 -0.11 -10.23 0.24 50% 26 1 5 0 5 2 5 2 1 3 1 7 7 4 5 3 4

65. Right sizing the healthcare workforce. Understanding the use of all healthcare providers. Typical we use the most expensive labor to handle health problems that could be care for by less expensive healthcare professionals. How we develop payment policies and healthcare regulations that allow health care delivery to deliver the highest quality care for the low costs. Scope of practice, the proper use of PAs and NPs, and understanding the role of pharmacies.

4.47 4.00 2.10 3.00 1.00 8.00 0.33 3.55 0.24 38% 54 4 5 0 3 3 7 2 1 3 3 8 8 4 5 6 5

66. Impact of new payment methodologies on healthcare markets. New methods are promoting integration to encourage risk sharing and reduce fragmentation to low the costs of care and improve quality. How does this change in market structure impact outcomes at the market level? What happens to prices, access, quantity and quality. What happens to small providers? Are some providers and insurers able to gain market power and use it to extract excess rents from consumers and payers?

4.13 4.00 1.92 5.00 1.00 8.00 0.20 5.30 0.20 38% 48 5 6 0 2 3 5 2 3 5 1 7 8 4 4 4 3

NOTES:

Standardized scores have been normalized using each rater's mean and standard deviation. This method adjusts for natural difference across raters in the anchoring points they use along the rating scale and in their tendency to use extreme values on the scale.

CV = Coefficient of Variation, a measure of dispersion among scores

ICC = Intra-class Correlation Coefficient, a measure of agreement among raters

% Below = Percent of raters who scored the item at or below the mean value



RANKED RESULTS: TOPICS ORDERED FROM LOW (MOST IMPORTANT) TO HIGH

(LEAST IMPORTANT) BASED ON THE STANDARDIZED MEAN SCORE

RWJF SYSTEMS FOR ACTION RESEARCH DELPHI PANEL RESULTS | Stage 3 Items Ordered by Mean Standardized Score (Lowest Score = Most Important)

Rank Item Mean Median St. Dev Mode Min Max Mean CV ICC %Below A B C D E F G H I J K L M N O P1 5. Financing and organizational models that promote collaboration across public health, healthcare, community

and social infrastructures (e.g., schools, social and community services, businesses) to improve important health issues, particularly addressing disparities in health and risk factors for health, across neighborhoods. Research is needed to identify incentives, pilot and align incentives across diverse community sectors, and determine effects on specific key health indicators. Health indicators should be those prioritized by community stakeholders including the most vulnerable, and mechanisms should include evidence-based and community-defined evidence especially for drivers that have less been the subject of at-scale intervention research (e.g., discrimination, marginalization).

1.69 1.00 1.14 1.00 1.00 5.00 -1.13 -0.59 0.06 94% 3 1 5 1 1 1 1 1 1 1 1 1 3 2 2 2

2 37. Can we demonstrate improved health outcomes (short and long term) when there is closer integration inthe health delivery system between health care and public health? Between mental health and health care? Between mental health and public health?

1.75 2.00 0.68 2.00 1.00 3.00 -1.12 -0.43 0.00 100% 2 2 2 1 2 1 1 1 1 1 2 2 3 2 3 2

3 16. Comparing the reach and effectiveness of public health services and/or policies to reduce and/or eliminate racial and ethnic disparities on health outcomes with the widest Black-white disparities, Latino-white disparities, and/or other subgroup disparities: http://aje.oxfordjournals.org/content/166/1/97.full.pdf+html

1.75 1.50 0.93 1.00 1.00 4.00 -1.10 -0.48 0.02 94% 1 1 3 1 1 1 2 1 3 1 2 2 1 4 2 2

4 6. Determine whether and how inter-sectoral collaboration between the health care system and social services (through effective referral systems for example) improve outcomes of care.

1.94 2.00 0.68 2.00 1.00 3.00 -0.94 -0.62 0.03 94% 2 1 1 2 2 2 2 3 2 1 3 1 3 2 2 2

5 38. Where population health is a shared responsibility between public health and health care, are healthoutcomes better?

2.50 2.00 1.32 2.00 1.00 5.00 -0.66 -1.09 0.08 81% 2 5 2 1 2 1 1 3 2 1 3 4 3 3 5 2

6 1. Examine the effectiveness of integrating a core set of community-based public health indicators intoelectronic health records in developing transitions of care and management strategies to improve outcomes and reduce the burden of chronic disease.

2.63 2.00 1.71 1.00 1.00 6.00 -0.56 -2.00 0.23 69% 1 1 2 1 5 1 3 1 2 2 2 2 5 3 6 5

7 45. Natural experiments of the impact of changes in local public health delivery system and/or financing onpopulation health.

2.69 3.00 1.20 3.00 1.00 5.00 -0.54 -0.92 0.01 88% 3 2 4 3 2 2 1 1 3 1 4 3 2 3 5 4

8 62. Achieving health equity and eliminating healthcare disparities. Identifying best practices and strategies thatimprove health outcomes for vulnerable populations. In particular, studying care delivery for racial and ethnic minorities, low income persons, persons with behavioral health problems, and chronically ill persons.

2.87 3.00 0.99 3.00 1.00 5.00 -0.50 -1.68 0.12 88% 2 3 0 2 3 3 5 1 4 2 4 2 3 3 3 3

9 63. Financial and other kinds of incentives in healthcare and their impact on access, costs and quality. Exploring and apply lessons learned from the role of behavioral economics and other models of human behavior from sociology and psychology to change the culture of health.

3.00 3.00 1.11 3.00 1.00 5.00 -0.50 -1.80 0.14 75% 3 2 0 2 3 4 2 2 4 1 4 3 4 5 3 0

10 54. How do community-level availability and reach of nonmedical public health programs (e.g. tobacco, obesity, injury, infectious disease control, food safety) and social programs (e.g. nutrition and food assistance, housing, income and child support) affect health insurance coverage and costs via Medicaid, Medicare, and private insurance (employer-based or exchange)? Are there public and private health insurance cost offsets attributable to nonmedical public health and social services, and if so how do offsets vary based on the extensiveness and intensiveness of available nonmedical services/programs?

2.81 2.00 1.91 2.00 1.00 8.00 -0.49 -1.90 0.16 75% 2 2 3 2 4 2 1 4 3 1 2 8 2 2 1 6

11 14. Compare the effectiveness of alternative policy and environmental change strategies to achieve long-term, sustained improvements in nutrition. Also, look at socioeconomic and racial/ethnic differences.

2.88 2.50 1.96 1.00 1.00 8.00 -0.45 -2.09 0.16 69% 2 4 3 1 4 1 1 2 5 2 3 8 1 5 1 3

12 21. Research examining and learning from positive deviance in improving local population health and reducing disparities in local population health outcomes, including how local stakeholders engage communities and use data to drive improvement.

2.81 3.00 1.33 2.00 1.00 5.00 -0.43 -2.00 0.13 75% 4 2 4 4 1 3 1 2 2 5 3 3 1 3 5 2

13 46. In communities with greater social investment is there better health status per dollar of healthexpenditures?

2.81 3.00 1.05 3.00 1.00 4.00 -0.42 -1.48 0.05 75% 3 4 4 4 3 2 4 1 3 1 2 3 3 2 4 2

14 61. Maximizing the value of the full continuum of care from self care, informal care, and various level of care provided by healthcare providers. Understanding how patient, caregivers, providers and payers can work together to get the most from our resources we are devoting to improving healthcare outcomes.

3.43 3.00 2.03 4.00 1.00 8.00 -0.42 -2.57 0.21 63% 0 4 0 3 2 4 2 1 4 1 7 8 3 4 3 2

15 50. Compare the population-level health and economic impact of alternative CMS-funded State InnovationModels (SIMs) that attempt to align payment systems across multiple health care payers and incorporate incentives and metrics for improving health status on a population-wide basis. (Note: CMS is not funding rigorous research on these SIM projects so there is a need for research).

3.20 3.00 1.78 3.00 1.00 6.00 -0.38 -2.84 0.21 63% 2 5 6 1 0 5 2 1 2 3 5 3 3 3 1 6

16 59. How can we address persistent discrimination at institutional, provider and individual levels to advance outcomes for under-resourced populations? What is the role of discrimination in undermining a culture of health? That is, how much is racism and discrimination impacting health outcomes and delivery of healthcare and public health services?

3.00 3.00 1.36 3.00 1.00 5.00 -0.36 -2.69 0.17 63% 3 4 0 3 1 2 3 4 5 1 4 3 2 5 4 1

17 49. Research that clarifies the added value of investments that communities make in intervening to improve health through different avenues, ranging from improving healthcare access and quality to improving education, poverty, addressing violence, etc., that is not only general (e.g., from the literature) but can be estimated and tailored for specific neighborhoods based on multi-level data on actual risk factors and health condition prevalence and structural characteristics of the zip code; both a model and mechanism to both estimate risk locally and estimate potential value of investments in action at environmental structural, system levels across diverse sectors.

2.94 3.00 0.85 2.00 2.00 4.00 -0.35 -1.42 0.01 81% 4 3 4 3 2 2 2 2 3 4 4 3 2 2 4 3

18 10. Research development among community stakeholders in collaboration with public health and healthcare practitioners, to have a community infrastructure to identify key needs, promote understanding of research among stakeholders, develop relationships with system and services researchers, and have resources to incentivize/encourage researchers to work with community stakeholders in addressing local and national culture of health, with strong inclusion of more vulnerable stakeholders in leading the community infrastructure for research (e.g., community centers of excellence for health research).

2.88 2.50 1.54 2.00 1.00 6.00 -0.35 -2.37 0.12 69% 3 2 4 3 1 4 2 3 5 1 5 1 2 2 6 2

19 30. Compare the effectiveness of changes to the built environment (e.g. city code, complete streets, urbanplanning) on health. Develop models for impact on health and economic impact.

2.88 2.00 1.82 2.00 1.00 8.00 -0.34 -3.35 0.24 69% 5 3 8 2 4 2 1 4 2 2 1 3 4 1 2 2

20 44. Identify recommended levels (per capita) of local and state investment in evidence-based public healthinterventions, and potential health and economic impacts.

3.13 3.00 1.96 3.00 1.00 9.00 -0.31 -2.36 0.08 75% 3 3 5 3 3 2 1 1 3 3 2 9 3 3 5 1

21 29. What is the impact of community development policies on health? Under what conditions are these policiesmost effective at improving health? What are the most important components of these policies?

2.94 3.00 1.44 3.00 1.00 7.00 -0.31 -2.89 0.14 69% 2 7 3 3 3 4 1 3 2 2 3 1 4 4 2 3

22 35. To what extent does the consideration of social information in the clinical encounter (such as through the inclusion of information on social determinants in the electronic health record) improve the outcomes of clinical care.

3.20 3.00 1.42 4.00 1.00 7.00 -0.28 -3.87 0.21 69% 4 7 2 0 3 3 4 4 2 2 4 1 3 4 3 2

23 51. Compare the health and economic impact of alternative ways of structuring and delivering housing assistance programs and policies through HUD, state and local housing authorities, and community redevelopment/reinvestment programs.

3.13 3.00 1.15 3.00 1.00 6.00 -0.24 -2.81 0.06 69% 2 4 6 3 2 4 4 3 3 1 3 4 3 3 2 3

24 13. Research to determine communication, engagement and motivational strategies to increase community andpublic awareness of equity, including dissemination of findings of the impact of community health indicators on the health of all and models for improving health of all by improving health of the most vulnerable. Within this research, there should be a focus on a comprehensive model of health spanning physical, emotional/psychological, and social well-being as well as on environmental and structural community drivers of health.

3.31 3.00 1.85 3.00 1.00 7.00 -0.16 -6.76 0.22 69% 3 5 6 3 1 2 5 1 3 5 4 3 1 2 7 2

25 11. Compare the effectiveness of alternative policy and environmental change strategies to achieve long-term, sustained increases in physical activity. Also, look at socioeconomic and racial/ethnic differences.

3.31 3.50 1.85 4.00 1.00 8.00 -0.15 -5.76 0.13 50% 5 4 3 4 4 2 1 1 5 2 3 8 1 4 2 4

26 64. The health care safety net in the age of Medicaid expansion and QHPs. How does the safety net evolve inera where most persons have coverage? What old problems persist? What new problem arise. What are the successes?

3.67 4.00 2.02 5.00 1.00 7.00 -0.11 -10.23 0.24 50% 1 5 0 5 2 5 2 1 3 1 7 7 4 5 3 4

27 48. Compare the effectiveness of combinations of evidence-based public health programs rather than isolatedinterventions to identify potential synergistic effects.

3.60 3.00 1.40 3.00 1.00 6.00 -0.09 -10.34 0.17 56% 5 3 3 3 4 2 1 2 5 5 5 6 3 0 3 4

Raw Scores Standardized Scores Individual Reviewer Ratings





1.69 1.00 1.14 1.00 1.00 5.00 -1.13 -0.59 0.06 94% 3 1 5 1 1 1 1 1 1 1 1 1 3 2 2 2


28 24. Research that includes in the building of healthier communities, a focus on prevention, early interventionfor social and structural risk factors for emotional well-being and long-term investment in development, particularly of children and families. For example, work on comprehensive public health models to address impact of trauma and prevent early consequences of child exposure to trauma or create safer environments to reduce trauma exposure, e.g., safe spaces in terms of violence, trauma, domestic violence -- areas that from a social determinants of health or prevention perspective, tend to be de-emphasized over many years as part of health promotion and interventions for social determinants.

3.44 4.00 1.79 4.00 1.00 7.00 -0.09 -9.42 0.14 44% 4 5 5 4 1 2 5 1 3 2 5 7 4 2 4 1

29 9. Are health outcomes better in jurisdictions in which health departments forge stronger and more numerous multi-sectoral partnerships?

3.53 3.00 2.03 2.00 1.00 8.00 -0.08 -18.95 0.34 63% 2 0 2 1 8 2 2 4 3 3 4 2 6 3 7 4

30 40. Research on the role of growing up in communities with high income inequality versus low income inequality on health outcomes, both contemporaneously and over the life course. Are certain public policies more effective than others in ameliorating the role of poverty and inequality on poor health of infants and adolescents?

3.56 3.00 2.28 1.00 1.00 9.00 -0.05 -21.85 0.20 50% 1 6 5 3 3 4 1 3 4 1 2 9 2 4 2 7

31 23. Research on the long-term health value of investments in healthier families and children, building resiliencyand coping, and resiliency to crises for individuals, families and communities, including in response to diverse disasters and environmental crises to community violence and major family traumatic events. That is, research that shows the value of particular and integrated strategies to invest in healthier families and children, both for short-term benefit and long-term value, either through natural experiments or long-term follow-up of community change efforts and especially their impact on children.

3.44 3.00 1.15 3.00 2.00 6.00 -0.04 -16.09 0.06 63% 4 3 5 4 3 2 4 2 3 5 3 6 3 3 3 2

32 60. Understanding opportunity and limits of the health care delivery system role in improving communityhealth. I thinking about healthcare providers working with providers of other social services to improve health. Understanding the benefits and also anticipating the unintended consequences.

3.60 4.00 1.40 2.00 2.00 6.00 -0.04 -23.66 0.17 50% 5 6 0 5 4 3 2 2 4 2 5 3 2 4 5 2

33 7. Compare the effectiveness of setting population health priorities using community health planning strategies developed in a health system alone versus a health system engaged with public health and community organizations.

3.50 3.50 1.37 4.00 1.00 6.00 -0.04 -14.44 0.02 44% 4 2 5 2 4 4 1 2 4 3 5 6 3 3 5 3

34 55. How do the prevalence and intensiveness of employer-supported and worksite-based health promotion and wellness programs affect community-level health and economic outcomes? Are there community-level spill-overs and positive externalities associated with employer-supported activities? Are there community-level network and peer effects related to the adoption, spread, and effectiveness of these activities among employers, employees, families, and communities?

3.44 3.50 1.41 4.00 1.00 6.00 -0.01 -53.58 0.10 69% 5 4 4 3 3 2 1 4 5 2 4 5 2 3 2 6

35 31. To what extent can community interventions or policy interventions not directly related to health enhance the effectiveness of traditional individual-level interventions (including health education as well as medical treatments)?

3.38 4.00 1.41 4.00 1.00 5.00 -0.01 -139.80 0.10 44% 3 5 5 4 2 1 1 4 4 2 5 2 4 5 3 4

36 56. What is the community economic impact resulted from the increased awareness of healthy life styles andbehavioral modification for the prevention of chronic illnesses and public health interventions (reduction of risk behaviors )?

3.80 3.00 1.82 3.00 1.00 7.00 0.00 -273.56 0.18 56% 5 5 0 3 3 2 1 2 3 3 7 7 3 6 4 3

37 25. Research on community engagement strategies, from relationship building to technology (as above) toincentives (above) and use of creative arts, empathy, to increase connection of people, increase motivation and commitment to equity in health outcomes and community building efforts. In addition, research may be needed on how the research process itself can be enhanced to have more community engagement as part of the design and conduct of research to build a culture of health.

3.56 4.00 1.93 4.00 1.00 8.00 0.00 -743.25 0.21 44% 3 2 4 5 1 4 4 3 4 6 5 8 1 2 4 1

38 8. Developing and testing interventions and methods to optimize the impact of organizational partnerships onimproving processes (accreditation aligned outcomes) in local public health systems.

3.63 3.50 1.89 3.00 1.00 7.00 0.03 37.95 0.19 44% 5 1 3 2 5 1 3 3 4 4 4 7 1 3 7 5

39 47. Compare the effectiveness and cost-effectiveness of evidence-based public health interventions to clinical interventions addressing the same issue.

3.63 3.50 1.50 4.00 1.00 7.00 0.03 19.16 0.06 50% 4 2 3 4 3 3 1 2 3 4 7 6 3 4 4 5

40 27. Paraprofessional training to advance health via communities where they live and work. For example, community health workers are provided technical training in health topics that they promote to community members. Potential research: standardization of technical training on a particular health topic or general of training hub for paraprofessionals.

3.69 3.50 1.74 3.00 1.00 8.00 0.09 11.09 0.17 50% 4 4 3 5 3 3 5 1 4 5 3 8 1 5 3 2

41 3. Research on approaches to system coordination and in particular electronic record environments thatfacilitate, rather than impede getting data out of systems, sharing data or using and analyzing data for system and/or community quality improvement efforts.

3.63 4.00 1.63 4.00 1.00 6.00 0.10 9.71 0.16 44% 4 1 3 4 2 3 2 1 4 4 6 3 5 4 6 6

42 52. How do federal, state, and local transportation projects, policies, and financing mechanisms impactpopulation health through pathways that include active commuting, injuries, and air quality? What are the net impacts, interactive and synergistic effects, and distributional effects on population subgroups stratified by SES, age, and geography (e.g. rural/suburban/urban)?

3.69 3.50 1.82 4.00 1.00 8.00 0.13 7.95 0.18 56% 4 4 8 4 3 3 1 4 3 2 3 5 4 2 2 7

43 36. Compare the effectiveness of alternative redesign strategies--using decision support capabilities, electronic health records, and personal health records--for increasing health professionals' compliance with evidence-based guidelines and patients' adherence to guideline-based regimens for chronic disease care.

3.75 4.00 1.98 4.00 1.00 9.00 0.13 7.37 0.17 50% 5 4 3 3 5 6 1 1 4 2 9 4 3 4 4 2

44 57. What programs are actually available and perceived as available in under-resourced areas and how does that compare to the perception of policymakers and healthcare and public stakeholders? How far have things gone (e.g., denial of programs have put children at risk for long-term health loss). How can 'public' programs actually become meaningfully available to impact under-resourced communities and what public commitment would that take?

3.93 3.00 2.09 3.00 1.00 8.00 0.13 9.83 0.30 50% 2 6 0 5 1 3 3 6 5 1 6 8 3 5 3 2

45 2. Evaluate a core set of public health system/population health measures to drive high quality public healthperformance, for their alignment with health care performance, and feasibility, usability, and importance in public health systems and services research

3.69 4.00 1.40 5.00 1.00 5.00 0.15 5.91 0.14 44% 5 3 4 1 4 1 4 3 5 2 3 5 5 4 5 5

46 39. Research on the intergenerational transmission of poverty on health outcomes, comparing residents inpersistently poor communities versus residents in non-poor communities. Related will be whether social services (access and use of both health and cash/in-kind transfers) mediate the transmission of poverty into poor health.

3.81 3.50 2.20 4.00 1.00 9.00 0.15 7.40 0.23 44% 1 6 4 4 5 3 3 3 5 2 1 9 4 2 2 7

47 33. Environmental health as it is related to environmental justice issues. This topic would incorporate issues ofinequity based on location of pollution and contamination as well as income, education, and ethnicity. This links to 'creating healthier, more equitable communities' mandates. Potential for research: 1) ethnographic study of the culture of pollution and 2) community based-research of environmental issues and links to community health.

3.88 4.00 2.06 2.00 1.00 8.00 0.20 6.10 0.26 44% 2 6 5 4 4 3 7 3 5 5 4 8 1 2 2 1

48 66. Impact of new payment methodologies on healthcare markets. New methods are promoting integration toencourage risk sharing and reduce fragmentation to low the costs of care and improve quality. How does this change in market structure impact outcomes at the market level? What happens to prices, access, quantity and quality. What happens to small providers? Are some providers and insurers able to gain market power and use it to extract excess rents from consumers and payers?

4.13 4.00 1.92 5.00 1.00 8.00 0.20 5.30 0.20 38% 5 6 0 2 3 5 2 3 5 1 7 8 4 4 4 3

49 4. Research to more systematically make available multiple level indicators for public health and healthcare practitioners (e.g., zip-code specific risk factors for major diseases, individual behavioral practices and healthcare access), to create both an infrastructure that promotes rather than inhibits using data from electronic systems; and makes available data to practitioners that is multiple-layered and contextualized. This will also require retraining of clinicians to use such layered data to inform public health and clinical practice.

3.75 4.00 1.34 4.00 2.00 6.00 0.22 3.85 0.13 44% 5 3 2 4 4 3 2 2 4 4 4 2 5 4 6 6





1.69 1.00 1.14 1.00 1.00 5.00 -1.13 -0.59 0.06 94% 3 1 5 1 1 1 1 1 1 1 1 1 3 2 2 2


50 19. Can we measure differences in resiliency between communities with greater integration of health and social services compared to those with lesser integration?

3.75 4.00 1.77 4.00 1.00 8.00 0.23 4.72 0.21 38% 3 4 6 4 8 1 3 4 4 1 5 2 5 3 3 4

51 53. Compare the population-level health and economic impact of alternative community benefit activities undertaken by tax-exempt hospitals as required by the enhanced IRS 990 regulations. How do the incidence, magnitude, and distribution of these impacts vary based on community benefit investment levels, based on institutional and community factors (including connectivity between hospitals and community organizations), and based on the processes used for community health needs assessment and priority-setting?

3.88 4.00 2.00 4.00 1.00 8.00 0.24 4.83 0.23 50% 4 3 4 2 6 2 5 4 5 1 3 7 4 3 1 8

52 15. What is the impact of sharing information on small area variations in health with local communities on their understanding of (1) health determinants or (2) the policies and actions that may be most effective to improve health?

3.94 4.00 1.18 4.00 2.00 6.00 0.31 2.63 0.11 44% 4 5 4 3 2 4 6 2 5 4 3 4 4 4 3 6

53 20. Can we measure differences in resiliency between communities with greater health equity compared tothose with lesser health equity?

4.00 4.00 1.75 4.00 1.00 7.00 0.32 2.85 0.15 25% 6 4 6 4 4 1 2 4 4 1 5 7 5 2 4 5

54 65. Right sizing the healthcare workforce. Understanding the use of all healthcare providers. Typical we use the most expensive labor to handle health problems that could be care for by less expensive healthcare professionals. How we develop payment policies and healthcare regulations that allow health care delivery to deliver the highest quality care for the low costs. Scope of practice, the proper use of PAs and NPs, and understanding the role of pharmacies.

4.47 4.00 2.10 3.00 1.00 8.00 0.33 3.55 0.24 38% 4 5 0 3 3 7 2 1 3 3 8 8 4 5 6 5

55 34. Role of multiple technical sectors, such as Agency for Toxic Substances and Disease Registry and the Environmental Protection Agency, contributing to the culture of health and environmental health of local communities.

4.06 4.00 1.12 4.00 1.00 5.00 0.36 2.29 0.11 25% 4 4 4 4 5 4 4 4 5 5 5 5 1 2 4 5

56 18. Advancing the use and examination of the utility of agent-based modeling, social network analysis, andother system science methods in improving population health. Examining the effectiveness and use of these methods and results to facilitate strategic and operational changes in public health organization and delivery through local public health and community stakeholder engagement.

4.19 4.00 1.38 5.00 2.00 7.00 0.45 1.79 0.11 25% 4 2 3 4 4 5 4 3 5 5 5 7 5 3 6 2

57 58. How can social media create, amplify, or replace community connections toward improved health? 4.47 5.00 1.81 5.00 1.00 7.00 0.47 2.56 0.26 19% 5 4 0 5 4 5 1 3 4 5 7 6 4 7 6 1

58 28. In communities that have adopted Health in All Policy approaches can we demonstrate improved health? 4.25 4.00 2.08 4.00 1.00 8.00 0.48 2.44 0.25 44% 4 8 3 5 6 3 2 4 2 1 7 4 4 6 7 2

59 17. What types of small area variations in health (and health inequalities across small areas) can be effectivelycharacterized using routinely and newly available data (including 'BIG data?) and state-of-the art methods?

4.25 4.00 1.29 5.00 2.00 7.00 0.49 1.43 0.07 31% 3 4 7 3 6 5 2 3 5 3 4 5 5 4 4 5

60 12. Research on the effectiveness of food and nutrition assistance (SNAP, WIC, school breakfast and lunch) andeducation (e.g. SNAP-Ed) on health outcomes. Study populations would include eligible participants and eligible nonparticipants, both cross-sectionally and longitudinally.

4.38 4.50 1.71 5.00 2.00 8.00 0.52 1.54 0.11 19% 2 4 5 5 4 6 3 3 5 2 6 8 4 5 2 6

61 41. Comparing the effectiveness of centralized, public health delivered services vs. decentralized, practice-baseddelivery of services on the reach and effectiveness in improving population health outcomes. This line of research would be comparing delivery in different settings / accountability structures.

4.50 4.00 1.75 4.00 1.00 8.00 0.58 1.67 0.17 31% 3 1 4 4 4 7 6 4 4 4 6 8 3 3 6 5

62 43. Linguistic and culturally appropriate approaches to translating health outcomes. Also, can also enveloptraditional practices that are customary in a culture. This would benefit the initiative's goal of improving health for marginalized populations. Potential for research on emerging areas in culturally based health practices.

4.69 5.00 2.06 5.00 1.00 8.00 0.69 1.44 0.18 19% 5 6 7 5 5 6 5 1 5 3 5 8 2 4 7 1

63 26. Rigorous research and evaluation is needed on how leaders are created and supported, and the progressionof their leadership, in culture of health pathways, across a range of leaders (community, patients, public health, providers, administrators, policymakers), which may inform future and current human capital programs.

4.69 4.50 2.15 5.00 2.00 9.00 0.70 1.36 0.17 25% 4 4 5 5 2 5 5 4 4 2 9 9 3 7 5 2

64 32. Is there a willingness at the community level (through taxes, programs, etc.) to support cross sectoral effortsto improve health and wellbeing?

4.81 5.00 2.14 6.00 2.00 9.00 0.80 1.14 0.15 31% 5 6 6 5 2 6 2 3 6 2 9 8 3 6 5 3

65 22. Research on the added value of diverse technology resources (e.g., social media and internetcommunication and intervention) to increase the efficiency of change in culture of health-related values of communities, systems and providers; to facilitate access to information for planners to create healthier environments; and to reinforce culture of health values over time and provide updates on mechanisms (whether individual or family coping or community improvement strategies), transparently available. Strategies should be available to increase the transparency concerning risk factors that apply to individuals and communities and strategies available and particularly if evidence-based, suitable for diverse languages, educational levels, from leaders to community members and patients.

4.88 5.00 1.67 5.00 1.00 8.00 0.90 0.92 0.12 19% 6 6 8 5 4 6 1 3 5 5 7 5 4 5 5 3

66 42. Development and testing of policies and interventions to improve workforce satisfaction among state andlocal public health workers (recent analyses of PHWINS data highlight major opportunities for improvement).

5.19 5.00 2.34 5.00 1.00 9.00 1.01 1.12 0.23 25% 5 3 7 5 4 3 5 3 6 5 8 9 1 7 9 3

NOTES:

Standardized scores have been normalized using each rater's mean and standard deviation. This method adjusts for natural difference across raters in the anchoring points they use along the rating scale and in their tendency to use extreme values on the scale.

CV = Coefficient of Variation, a measure of dispersion among scores

ICC = Intra-class Correlation Coefficient, a measure of agreement among raters

% Below = Percent of raters who scored the item at or below the mean value

12 Systems for Action National Program Office--Delphi Panel Stage 3 Results

Systems for Action is a National Program Office of the Robert Wood Johnson Foundation and a collaborative effort of the Center for Public Health Systems and Services Research in the College of Public Health, and the Center for Poverty Research in the Gatton College of Business and Economics, administered by the University of Kentucky, Lexington, Ky.

© 2015 Systems for Action. All rights reserved. The University of Kentucky is an Equal Opportunity Employer.

Center for Public HealthSystems and Services Research

systems for actionsystemsforaction.org/sites/default/files/resource_files/rwjf s4a... · 2....

Documents