annual report - academie catholique de france · adam bosworth keas, inc. david r. brennan...
TRANSCRIPT
AnnuAl RepoRt
Authored by Institute of Medicine staff, the text summarizes work and discussions over the past two years by the Roundtable on Evidence-Based Medicine. Content is drawn from publications and discussions of the Roundtable’s meetings and workshops.
The Roundtable on Evidence-Based Medicine
Learning Healthcare System
Concepts v. 2008
AnnuAl RepoRt
ii Institute of Medicine
Denis A. Cortese, M.D. (Chair) Mayo Clinic
Adam Bosworth Keas, Inc.
David R. Brennan AstraZeneca PLC
Carolyn M. Clancy, M.D. Agency for Healthcare Resrch & Quality
Helen Darling National Business Group on Health
James A. Guest Consumers Union
George C. Halvorson Kaiser Permanente
Carmen Hooker Odom Milbank Memorial Fund
Michael M.E. Johns, M.D. Emory University
Michael J. Kussman, M.D., M.S. Department of Veterans Affairs
Cato T. Laurencin, M.D., Ph.D. University of Virginia
Stephen P. MacMillan Stryker Corporation
Mark B. McClellan, M.D., Ph.D. The Brookings Institution
Elizabeth G. Nabel, M.D. National Heart, Lung & Blood Institute
Mary D. Naylor, Ph.D., R.N.University of Pennsylvania
Peter NeupertMicrosoft Corporation
Nancy H. Nielsen, M.D., Ph.D. American Medical Association
Jonathan B. Perlin, M.D., Ph.D. HCA, Inc.
Richard Platt, M.D., M.S.Harvard Medical School
John Rother, J.D.AARP
Tim Rothwellsanofi-aventis U.S.
John W. Rowe, M.D.Columbia University
Donald M. Steinwachs, Ph.D.Johns Hopkins University
Andrew L. SternSEIU
I. Steven Udvarhelyi, M.D.Independence Blue Cross
Frances M. ViscoNational Breast Cancer Coalition
Kerry N. WeemsCntrs for Medicare & Medicaid Srvcs
William C. WeldonJohnson & Johnson
Janet Woodcock, M.D.Food & Drug Administration
ROUNDTABLE MEMBERS
Learning Healthcare System Concepts iii
Peter NeupertMicrosoft Corporation
Nancy H. Nielsen, M.D., Ph.D. American Medical Association
Jonathan B. Perlin, M.D., Ph.D. HCA, Inc.
Richard Platt, M.D., M.S.Harvard Medical School
John Rother, J.D.AARP
Tim Rothwellsanofi-aventis U.S.
John W. Rowe, M.D.Columbia University
Donald M. Steinwachs, Ph.D.Johns Hopkins University
Andrew L. SternSEIU
I. Steven Udvarhelyi, M.D.Independence Blue Cross
Frances M. ViscoNational Breast Cancer Coalition
Kerry N. WeemsCntrs for Medicare & Medicaid Srvcs
William C. WeldonJohnson & Johnson
Janet Woodcock, M.D.Food & Drug Administration
ThE IOM ROUNDTABLE ON EvIDENcE-BASED MEDIcINE
The Institute of Medicine’s Roundtable on Evidence-
Based Medicine was established in 2006, in the face of
growing awareness that care that is important is often not
delivered, and care that is delivered is often not important—
with significant health and economic consequences for
Americans. The Roundtable is composed of leadership from
key sectors—patients, health providers, payers, employers,
manufacturers, health information technology, researchers,
and policy makers—united in their commitment to work
together on innovative approaches to transform the way
evidence on clinical effectiveness is developed and used to
improve health and health care in the United States. It aims
to marshal emerging technology and expertise to create
a learning healthcare system in which each patient care
experience naturally reflects the best available evidence,
and, in turn, adds seamlessly to learning what works best in
different circumstances. Activities of the Roundtable include
public meetings and seminars on key topics, as well as
collaborative joint projects for innovation on central issues.
iv Institute of Medicine
ROUNDTABLE chARTER AND vISION STATEMENT
The Institute of Medicine’s Roundtable on Evidence-Based Medicine has
been convened to help transform the way evidence on clinical effectiveness is
generated and used to improve health and health care. We seek the development
of a learning healthcare system that is designed to generate and apply the best
evidence for the collaborative healthcare choices of each patient and provider;
to drive the process of discovery as a natural outgrowth of patient care; and to
ensure innovation, quality, safety, and value in health care.
Vision: Our vision is for a healthcare system that draws on the best evidence to
provide the care most appropriate to each patient, emphasizes prevention and
health promotion, delivers the most value, adds to learning throughout the
delivery of care, and leads to improvements in the nation’s health.
Goal: By the year 2020, ninety percent of clinical decisions will be supported
by accurate, timely, and up-to-date clinical information, and will reflect the
best available evidence. We feel that this presents a tangible focus for progress
toward our vision, that Americans ought to expect at least this level of
performance, that it should be feasible with existing resources and emerging
tools, and that measures can be developed to track and stimulate progress.
Context: As unprecedented developments in the diagnosis, treatment, and
long-term management of disease bring Americans closer than ever to the
promise of personalized health care, we are faced with similarly unprecedented
challenges to identify and deliver the care most appropriate for individual
needs and conditions. Care that is important is often not delivered. Care that
is delivered is often not important. In part, this is due to our failure to apply
the evidence we have about the medical care that is most effective—a failure
related to shortfalls in provider knowledge and accountability, inadequate
care coordination and support, lack of insurance, poorly aligned payment
incentives, and misplaced patient expectations. Increasingly, it is also a result
of our limited capacity for timely generation of evidence on the relative
effectiveness, efficiency, and safety of available and emerging interventions.
Improving the value of the return on our healthcare investment is a vital
imperative that will require much greater capacity to evaluate high priority
clinical interventions, stronger links between clinical research and practice,
and reorientation of the incentives to apply new insights. We must quicken
our efforts to position evidence development and application as natural
outgrowths of clinical care—to foster health care that learns.
Learning Healthcare System Concepts v
Approach: The IOM Roundtable on Evidence-Based Medicine serves as
a forum to facilitate the collaborative assessment and action around issues
central to achieving the vision and goal stated. The challenges are myriad
and include issues that must be addressed to improve evidence development,
evidence application, and the capacity to advance progress on both dimensions.
To address these challenges, as leaders in their fields, Roundtable members
will work with their colleagues to identify the issues not being adequately
addressed, the nature of the barriers and possible solutions, and the priorities
for action, and will marshal the resources of the sectors represented on the
Roundtable to work for sustained public-private cooperation for change.
Activities include collaborative exploration of new and expedited approaches
to assessing the effectiveness of diagnostic and treatment interventions, better
use of the patient care experience to generate evidence on effectiveness,
identification of assessment priorities, and communication strategies to
enhance provider and patient understanding and support for interventions
proven to work best and deliver value in health care.
Core concepts and principles: For the purpose of the Roundtable activities,
we define evidence-based medicine broadly to mean that, to the greatest extent
possible, the decisions that shape the health and health care of Americans—by
patients, providers, payers, and policy makers alike—will be grounded on
a reliable evidence base, will account appropriately for individual variation
in patient needs, and will support the generation of new insights on clinical
effectiveness. Evidence is generally considered to be information from clinical
experience that has met some established test of validity, and the appropriate
standard is determined according to the requirements of the intervention
and clinical circumstance. Processes that involve the development and use of
evidence should be accessible and transparent to all stakeholders.
A common commitment to certain principles and priorities guides the
activities of the Roundtable and its members, including the commitment
to: the right health care for each person; putting the best evidence into
practice; establishing the effectiveness, efficiency, and safety of medical care
delivered; building constant measurement into our healthcare investments;
the establishment of healthcare data as a public good; shared responsibility
distributed equitably across stakeholders, both public and private; collaborative
stakeholder involvement in priority setting; transparency in the execution of
activities and reporting of results; and subjugation of individual, political, or
stakeholder perspectives in favor of the common good.
vi Institute of Medicine
ThIS SUMMARy
Learning Healthcare System Concepts v. 2008 presents a summary of important touchstones identified during the first two years of the work of the IOM Roundtable on Evidence-Based Medicine. It contains no consensus recommendations, and has been assembled by Roundtable staff from the common themes and discussion summaries of the various meetings, as noted below.
Section 1: background material prepared for the Roundtable website (www.iom.edu/ebm-effectiveness) and from the staff issue overview, Learning What Works Best: The Nation’s Need for Evidence on Comparative Effectiveness in Health Care (2007).
Section 2: figures from the Roundtable workshop, The Learning Healthcare System, July 20-21, 2006; the IOM Annual Meeting, Evidence-Based Medicine and the Changing Nature of Health Care, October 8, 2007; and the Roundtable workshop, Redesigning the Clinical Effectiveness Research Paradigm, December 12-13, 2007.
Section 3: from the Roundtable workshop, The Learning Healthcare System, July 20-21, 2006.
Section 4: from the Roundtable workshop, Leadership Commitments to Improve Value in Health Care, July 23-24, 2007, and the follow-up Roundtable sectoral coordinators meeting, December 3, 2007.
Section 5: from the Roundtable workshop, Judging the Evidence, February 5, 2007.
Section 6: from the Roundtable staff issue overview, Learning What Works Best: The Nation’s Need for Evidence on Comparative Effectiveness in Health Care (2007).
Section 7: from the Roundtable workshop, Redesigning the Clinical Effectiveness Research Paradigm, December 12-13, 2007.
Learning Healthcare System Concepts vii
Section 8: from the Roundtable workshop, Clinical Data as the Basic Staple of Health Learning, February 28-29, 2008.
Section 9: from the Roundtable workshop, Leadership Commitments to Improve Value in Healthcare, July 23-24, 2007, follow-up Sectoral Coordinators meeting, and the Roundtable Members meeting, September 24, 2007.
Section 10: from the Roundtable workshop, Leadership Commitments to Improve Value in Healthcare, July 23-24, 2007.
Like the dynamic nature of evidence itself, the issues, concepts, and priorities noted in Roundtable activities will continue to evolve, improve, and be revised as indicated—hence the v. 2008 modifier. Source material and related background material may be accessed at the Roundtable’s website: http://www.iom.edu/ebm. These include the following publications, available through the National Academies Press:
• The Learning Healthcare System (2007)• Leadership Commitments to Improve Value in Health
Care: Finding Common Ground (2008—in press)• Redesigning the Clinical Effectiveness Research
Paradigm: Innovation and Practice-Based Approaches (2008—in press)
• Clinical Data as a Basic Staple of Health Learning: Creating and Protecting a Public Good (2008—in press)
Publications planned for 2009 include:
• Engineering a Learning Healthcare System: A Look at the Future (2009—in preparation)
• Learning What Works: Infrastructure and Clinical Priorities for Learning Which Care Is Best (2009—in planning)
• Making Better Choices: Getting the Value We Deserve from Health Care (2009—in planning)
“We seek the development of a learning healthcare system
that is designed to generate and apply the best evidence
for the collaborative health care choices of each patient
and provider; to drive the process of discovery as a natural
outgrowth of patient care; and to ensure innovation,
quality, safety, and value in health care.”
—Charter, IOM Roundtable on
Evidence-Based Medicine
LEARNINg hEALThcARE SySTEM cONcEpTS v. 2008
1. Imperative: Patient centered, better results,
better value, and the changing scene
2. context: Some illustrative context
3. Learning: The learning healthcare system
4. value: Advancing the value proposition
5. Evidence: Evidence stewardship in a
learning healthcare system
6. capacity: Expanding the effort to learn
what works
7. Research: Redesigning the clinical effectiveness
research paradigm
8. Data: Creating a clinical data utility for
continuous improvement
10. commitments: Leadership commitments
to better value in health care
9. Expectations: Responsibilities and
expectations for participants
Learning Healthcare System Concepts �
1 IMpERATIvEPatientcentered,betterresults,bettervalue,andthechangingscene
The United States faces two fundamental healthcare
challenges: (a) meeting the moral obligation to
ensure that everyone is secure in his or her ability to
receive the basic prevention and treatment services needed;
and (b) ensuring that the nation—and each of us—receives
the value we deserve from the healthcare investments we
make. As a nation, we fall shortest on the value proposition.
The immediate and dominant issue of greatest threat to the
health and economic security of Americans is the failure
of our healthcare system to deliver the value that should
be expected from the health care we receive. The IOM
Roundtable on Evidence-Based Medicine is devoted to
accelerating the development of a healthcare system with
an inherent capacity for continuous improvement in the
effectiveness and efficiency of care delivered.
Health care in the United States currently underperforms
on many dimensions. From the global perspective, with per
capita expenditures more than 20 percent higher than any
other country in the world and more than twice the average
expenditure for European countries, the nation ranks well
below at least two dozen others on key health indices such
as infant survival and life expectancy. In part, this is because
people too often do not receive the care they need. One study
found that, where evidence exists, only a little more than half
of recommended services were actually delivered. In part, it
is also because the services people receive are too often not
always the ones necessary or right for them. The intensity of
services for similar conditions—as with procedures for which
discretion is at play, for example, back surgery, hysterectomy,
and bypass surgery—can be many times higher in one place
� Institute of Medicine
than another, without any difference in outcome or results.
For example, 4.6 lumbar fusions were reported per 1,000
Medicare enrollees annually in Idaho Falls, Idaho, compared
with 0.2 in Bangor, Maine, with no difference in results.
Clearly, more does not by itself equate to better—and the
variation is greater for conditions in which the evidence is
more limited.
One estimate suggested that only about a quarter of the
discrepancy in Medicare spending across regions could be
explained by population illness levels, and if all regional
spending levels matched those of the lowest, Medicare
could see savings of up to $40 billion (1996 dollars) without
compromising health status. This is not just an issue of
waste, but also of potential harm. In To Err is Human, the
IOM Committee on the Quality of Health Care in America
found that each year between 44,000 and 98,000 Americans
die due to medical errors.
It is clear that medical care decision making is already
strained, at both the level of the individual patient and the
level of the population as a whole, by the growing number
of diagnostic and therapeutic options for which evidence is
insufficient to make a clear choice. The strain will worsen
as choices grow, with new pharmaceuticals, medical devices,
diagnostics, biologics, and procedures. A recent review
suggests that half or more of the growth in medical spending
in recent years is attributable to change in technology.
Compounding the complexity is the fact that the world of
health care is about to experience dramatic new insights
on the genetic variation in individual response to different
diagnostic and treatment interventions. The age of
personalized medicine will soon be a reality, if the capacity
can be developed to contend with its insights. Today, the
Learning Healthcare System Concepts �
IMpER
ATIv
E
average clinical encounter already requires a health provider
to manage more variables than would be considered
reasonable given what is known about the capabilities of the
human mind. With changes in knowledge and options, over
the next decade, that same encounter will require managing
variables perhaps an order of magnitude greater in number.
Engaging ways to address these issues and their implications
has been the focus of the activities of the IOM Roundtable
on Evidence-Based Medicine over the past two years.
Because of the centrality of cross-sector dialog to effective
action on many of these issues, especially important to the
collaborative spirit has been the input of the sector working
groups, which include Roundtable members and beyond.
Summarized here are the products of Roundtable meetings
and sectoral discussions, workshops, commissioned pieces,
and staff issue papers.
Learning Healthcare System Concepts �
2 cONTEXTSomeillustrativecontext
In the past two years, the IOM Roundtable on Evidence-
Based Medicine has met five times and has sponsored
six large public workshops and meetings and several
smaller meetings on issues central to evidence-driven health
care. Participants in these activities are listed at the back of
this booklet. In the course of the various meetings, many
important points have been made in poignant graphic
format. Presented below are ten representative examples
that illustrate the compelling context of our activities. None
of these tells the whole story; each gives a sense of both
opportunity and challenge.
cONSTANTLy chANgINg MEDIcAL INTERvENTIONS
Molly Coye, IOM Meeting, 8 October 2007. Challenges of increasing diversity, complexity, and cost of interventions.
S-1 new.eps
Challenge = current evidence paradigms
Rapid evolution of technologies
Need for guidance documents for evidence development
Diversity, complexity and cost rather than quantity� Pharmaceuticals biotech drugs � Diagnostics pharmacogenomics and biomarkers� Information technology� Hybrid technologies: devices + pharmaceuticals + information technology � “Follow-on biologics” and biologic equivalency
�
�
Ongoing, rapid-cycle innovation
� Institute of Medicine
InsIghts Into IndIvIdual varIatIon
David Altshuler, IOM Meeting, 8 October 2007. Rapid increases in identification of gene variants for common diseases.
PaucIty of evIdence
Robert Califf, IOM Meeting, 12 December 2007. Less than 20% of AHA/ACC heart disease management recommendations are based on a high level of evidence and over 40% are based on the lowest level of evidence: Level A evidence (multiple populations and risk strata) to Level C (very limited population risk strata). The proportion of recommendations with high evidence levels has not increased over time.
new S-3.eps
Fac
ts p
er D
ecis
ion 1000
10
100
5Human
Cognitive Capacity
2000 20101990 2020
Structural Genetics: e.g. SNPs, haplotypes
Functional Genetics: Gene expression
profiles
Proteomics and othereffector molecules
Decisions by Clinical Phenotype
new 2-7.eps
$7,200 to 11,600 (74)6,800 to < 7,200 (45)
6,300 to < 6,800 (55)5,800 to < 6,300 (60)
4,500 to < 5,800 (72)Not Populated
Learning Healthcare System Concepts �
William Stead, IOM Meeting, 8 October 2007. Growth in facts affecting provider decisions versus human cognitive capacity.
consequences for PractIce
Elliot Fisher, IOM Meeting, 8 October 2007. Wide geographic variation in per capita Medicare spending, 2003. [The numbers in parentheses indicate the number of regions in each group.]
con
text
challenges to ProvIders
new 2-6.eps
1966 1972 1978 1984 1990 1996 2002 2008 2014 2020 2026 2032 2038 2044 2050
Per
cent
of G
ross
Dom
estic
Pro
duct
0
5
10
15
20
25Actual Projection
2.5 Percentage Points
Differential of:
1 Percentage PointBeginning in 2025
1 Percentage PointBeginning in 2015
10 Institute of Medicine
consequences for value
Peter Orszag (citing Dartmouth), IOM Meeting, 8 October 2007. Inverse relationship between per capita Medicare spending and hospital quality ranking, 2000-2001.
Peter Orszag, IOM Meeting, 8 October 2007. Scenarios for growth in share of GDP for Medicare and Medicaid spending.
ImPlIcatIons for the economy
3--1
By opening on the PC we got the type going the right way,but in this case each letter is a separate object
0
5
10
15
20%
dia
beti
cp
atie
nts
with
HgA
1c>
9
Jun
99Sep Dec
Mar
00Ju
nSepDec
Mar
01Ju
nSep Dec
Mar
02Ju
nSepDec
Mar
03Ju
nSep Dec
Mar
04Ju
nSepDec
Mar
05Ju
nSep Dec
Mar
06
0
5
10
15
20
16.6
15.4
14.5
14.4
13.8
13.1
12.3 12
.7
12.1
10.8
9.59.
2 9.4
8.6
8.2
8 8.3
7.9
7.5 7.
7 7.8
7.6
7.6
7.6 8.
37.
87.3
7.9
new S-4.eps
THEN• Monitoring• MD exam
• Nurse observations
THEN NOW TOMORROW
NOW• Radiology• EKG
• Labs• Meds• Monitoring
• MD exam• Nurse observations
TOMORROW• Continuous location tracking
• Constant vital signs monitoring• Transcutaneous monitoring of
glucose, ETOH, etc.• Noninvasive testing• Exhaled breath analysis
• Testing to maintain wellness• Increased image utilization• Genomic data
• Proteomics• Personalized pharmacy• Radiology• EKG• Labs
• Meds• Monitoring• MD exam
• Nurse observations
Learning Healthcare System Concepts 11
BRINgINg LEARNINg TO pRAcTIcE
Brent James, IOM Meeting, 20 July 2006. Improvements in diabetes control with systematic clinical program management.
Peter Neupert, IOM Meeting, 8 October 2007. Growth in clinical information and the role for IT in its management.
cON
TEXT
cLINIcAL INfORMATION gROwTh
12 Institute of Medicine
fORcES IN EvIDENcE-DRIvEN hEALTh cARE
Robert Hayward, IOM Meeting, 8 October 2007. Social and technical elements to address in evidence-driven care.
Integration
Social
Technical
CultureNetworks
Context
EvidenceSynthesis
DisseminationTechnology
SkillsandKnowledge
Collaboration
}{
Learning Healthcare System Concepts 1�
3 LEARNINgThelearninghealthcaresystem
Contending with the changes in health care clearly
requires doing things differently, as discussed at the
Roundtable’s workshop, The Learning Healthcare
System. The notion of dependence on expensive, time-
consuming, and relatively narrow clinical experiments
as a static and sole source of evidence on the constantly
accelerating flow of diagnostic and treatment challenges,
is unfeasible. Similarly impractical is the prospect of
individual caregivers trying to keep up on their own with
rapidly changing knowledge needs. A learning healthcare
system is needed, in which new insights and evidence are
constantly produced, and value is constantly enhanced, by
real-time learning from the clinical experience and seamless
application of the lessons in the care process.
The IOM report Crossing the Quality Chasm found that if
health care is to meet its central aims to be safe, effective,
patient-centered, timely, efficient, and equitable, several
elements must be central features of the care process:
(1) care based on continuous healing relationships; (2)
customization based on patient needs and values; (3) the
patient as the source of control; (4) shared knowledge and
the free flow of information; (5) evidence-based decision
making; (6) safety as a system property; (7) transparency;
(8) anticipation of needs, (9) continuous decrease in waste,
and (10) cooperation among clinicians.
Achieving health care centered on these principles requires
a transformative learning and change process—one that
takes full advantage of the tools presented with emerging
information technology and builds the continuous
improvement necessary for the effectiveness, efficiency,
1� Institute of Medicine
and sensitivity inherent in bringing the right care to the
right patient at the right time. Fostering progress toward
this vision, beginning with the basic challenge of knowing
what the right care is for each person, is the aim of the IOM
Roundtable on Evidence-Based Medicine. And, to identify
and explore the necessary foundation stones on which to
build, the Roundtable’s first public meeting focused on the
elements of what it has termed the learning healthcare system.
Noted below are several key features of such a system.
• Continuous improvement in the value delivered.
A learning healthcare system is one that maintains
a constant focus on the health and economic value
returned by care delivered, and continuously improves
in its performance.
• Learning in health care as a partnership enterprise.
Broad culture change is needed to enable the evolution
of the learning environment as a common, partnership
cause of patients, providers, and researchers alike.
The Learning healthcare System• Continuous improvement in the value delivered• Learning in health care as a partnership enterprise• Developing the point of care as the knowledge engine• Full application of information technology• Database linkage and use• Advancing clinical data as a public utility• Building innovative clinical effectiveness research into practice • Patient engagement in the evidence process• Development of a trusted scientific intermediary • Leadership that stems from every quarter
Learning Healthcare System Concepts 1�
• Developing the point of care as the knowledge engine.
Given the rate at which new interventions are developed,
along with new insights about individual variation in
response to interventions, the point of care must be the
central focus for the continuous learning process.
• Full application of information technology. The rate of
learning—both the application and development of
evidence—will depend on the full and strategic
application of information technology, including
electronic health records central to long-term change.
• Database linkage and use. The emergence of large,
electronically based datasets offer important new
sources for quality improvement and evidence
development. Progress requires fostering interoperable
platforms, linking analyses, establishing networks, and
developing new approaches for ongoing searching of
those databases for patterns and clinical insights.
• Advancing clinical data as a public utility. Meeting the
potential for using new datasets as central sources of
evidence on the effectiveness and efficiency of medical
care, will require recognition of their qualities as a public
good, including assessing issues related to ownership,
availability, and use for real-time clinical insights.
• Building innovative clinical effectiveness research
into practice. Improving the speed and reliability of
evidence development requires fostering development
of a new clinical research paradigm—one that deploys
careful criteria for trial conduct, draws clinical research
more closely to the experience of clinical practice,
advances new study methodologies adapted to the
practice environment, and engages cultural incentives
to foster more rapid learning.
• Patient engagement in the evidence process. Accelera-
ting the potential for better development and application
LEARN
INg
16 Institute of Medicine
of evidence requires improved communication between
patients and healthcare professionals about the nature
of the evidence base, and the need for partnership in its
development and use.
• Development of a trusted scientific intermediary.
Greater synchrony, consistency, and coordination
in the priority setting, development, interpretation,
and application of clinical evidence requires a trusted
scientific intermediary to broker the perspectives of
different parties.
• Leadership that stems from every quarter. Strong,
visible, and multifaceted leadership from all involved
sectors is necessary to marshal the vision, nurture the
strategy, and motivate the actions necessary to create
the learning healthcare system we need.
Learning Healthcare System Concepts 1�
4 vALUEAdvancingthevalueproposition
Also established as an early Roundtable priority was
development of a stronger focus on the value we
get from the health care delivered. In fall 2007, the
Roundtable’s Sectoral Strategies working group developed
an initial summary of elements of the value proposition,
described below as touchstones for further exploration and
characterization of the elements of value.
• Value is expressed as what we gain relative to what we
give up—the benefit relative to the cost.
• Value in health care is expressed as the physical health
and sense of well-being achieved relative to the cost.
• High value in health care means getting the right care to
the right patient at the right time for the right price.
• Value from pooled arrangements is expressed as the
aggregate gains relative to the aggregate costs.
• Determining the value proposition for a particular
intervention therefore begins with understanding its
safety, effectiveness, and cost in different populations
and circumstances, relative to alternatives, and its
potential as a source of high-value innovation.
• In assessing value, benefits and costs—often multifaceted
—are objectively characterized and measured.
value in health care value in health care
• The right care• To the right patient• At the right time• For the right price
1� Institute of Medicine
Each of us seeks value in what we do. At the most basic level,
this means adding to our lives in ways that are meaningful to
us. For any given person, the nature and magnitude of value
gained is personal, and varies by circumstance and what
we must give up for the gain. Meaning may be found in our
abilities, our choices, our relationships, our security, our sense
of well-being, our possessions, our environments, and so on.
Enhancing our status on any relevant dimension may require a
measure of diminished status on another—for example, time,
energy, money. Value is expressed as what we gain relative to
what we give up—the benefit relative to the cost.
The purpose of health care is to enhance physical and mental
health status. Value in health care is expressed as the physical
health and sense of well-being achieved relative to the cost. High
value in health care therefore means getting the right care at the
right time to the right patient for the right price. As with other
issues, value in health care—what is “right” about care, time,
and price—depends on vantage point and circumstance.
Perceived value will vary according to one’s view as a patient,
caregiver, family member, neighbor, community leader,
employer, healthcare manager, innovator, or policy official.
Sometimes the determination of value is complicated by the
fact that a benefit received is the result of a cost shared or
borne elsewhere. This is typical of activities in which there is
advantage or necessity to arrange for pooling of resources to
provide the benefit. Education, police, highways, parks, even
manufacturing and information services, are all examples
of circumstances in which benefits—such as services or
goods—are designed through common processes to make
it possible for groups of people to benefit. In these cases,
the gain is considered from two perspectives: the individual
gain for one person’s investment and the social gain from
Learning Healthcare System Concepts 1�
the collective investment. Value from pooled arrangements is
expressed as the aggregate gains relative to the aggregate costs.
The challenge of measuring value in any arena lies in
identifying, characterizing, and assessing the nature of the
gains and costs. This is especially complicated in health
care, given the difficulty of measuring inputs and outcomes,
and because payment for health care is primarily organized
through risk-spreading insurance mechanisms. Value must be
understood from both individual and societal or population
perspectives—which may be at substantial variance.
Determining the value proposition for a particular intervention
therefore begins with understanding its safety, effectiveness,
and cost in different populations and circumstances, relative
to alternatives, and its potential as a source of high-value
innovation. From this point, certain interventions can clearly
be assumed to improve the overall value from health care if
properly applied—for example, those that offer substantially
better safety, outcomes, or patient satisfaction with marginal
changes in (economic and other) costs; those that can
offer improvements for conditions that previously had no
alternatives, at costs comparable to the management of other
conditions; those that are cost reducing or otherwise improve
the overall efficiency of health care. Other interventions may
require more detailed analyses, including using metrics
that facilitate comparisons on effectiveness, factoring
patient preferences and innovation potential into the value
calculation, and developing decision rules on balancing value
to individuals with aggregate value to society. Whatever rules
and determinations are made will require constant updating
and refining as new information becomes available on the
targeting, impact, potential, and costs of interventions.
vA
LUE
20 Institute of Medicine
Rewarding caregivers who deliver high-value care should
be a central goal of incentives embedded in healthcare
financing. Accomplishing this aim will require analytic
tools and capacity beyond what are currently available,
including development of the capacity to study relative
safety and effectiveness; to inform, assess, and integrate
patient preferences; to better characterize and target groups
at particular risk; to understand and balance the various
elements of cost; to fashion the principles needed to ensure
an appropriate balance between an individual’s value
proposition and that of the aggregate for a population; to
systematically track the results of healthcare interventions;
and to identify the system elements most conducive to high-
value health care.
Learning Healthcare System Concepts 21
5 EvIDENcEEvidencestewardshipinalearninghealthcaresystem
The effectiveness and efficiency of health care begins
with the evidence, the focus of the second Roundtable
workshop, Judging the Evidence. Innovation is the
engine of progress, and evidence is its linchpin. Participants
noted that because the care, integrity, and timeliness with
which medical evidence is produced, interpreted, and applied
is so fundamental to identifying, confirming, and improving
on best practices for different circumstances, it is the shared
responsibility of all stakeholders.
Without—a mutual understanding among all parties
involved—intervention developers, regulators, payers,
healthcare organizations, providers, patients—of the basis
on which evidence will be interpreted and applied, the
generation of new evidence may result in little or no benefit
to patient care. Examples of the issues mentioned included
the notions that payers may lack consistency in their coverage
decisions, regulators may seem arbitrary in their requests,
health product companies may feel that ambiguous
requirements and decision criteria constrain innovation,
caregivers may be uncertain about the appropriate
intervention, and patients may feel confused about what’s
best for them. All of these dynamics impede progress,
efficiency, and the results that should be expected from our
healthcare delivery system.
Achieving a learning healthcare system requires robust
capacity to generate and apply new insights based upon
clarity and confidence in the principles that serve as a
foundation and guide to the generation and use of the
evidence. Participants in the Roundtable’s sectoral strategies
collaboration offer the following set of core principles to
22 Institute of Medicine
guide stewardship of evidence in a learning healthcare
system. Individuals and institutions involved in developing
or applying evidence on the effectiveness of a healthcare
diagnostic or treatment product, procedure, or service are
expected to follow these principles in their work.
• Independent and trusted source. Conclusions and
recommendations based on the evidence must derive
from a trusted organization of proven independence in
the conduct of evidence assessment and validation.
• Evaluated by established standards. Each diagnostic
or treatment product, procedure, or service used in
health care must be supported by evidence of safety
and effectiveness produced through established and
validated standards and processes. If circumstances
preclude customary assessment, the intervention must
be supported by ample documentation of biologic and
clinical plausibility.
• Collaborative and consultative review. Strategies
and activities to assess the evidence for a particular
intervention, and apply the results in advancing the
principles for Interpretation and Use of clinical Evidence
principles for Interpretation and Use of clinical Evidence
• Independent and trusted source • Evaluated by established standards • Collaborative and consultative review • Transparent decision processes • Appropriately applied • Patient-focused choices• Effective communication and comprehension• Value driven • Accountability and assessment• Continuous learning
Learning Healthcare System Concepts 2�
most effective health care, should be collaborative,
consultative, and supportive of the most efficient use of
limited resources.
• Transparent decision processes. Each decision to
propose, introduce, use, and pay for a diagnostic or
treatment product, procedure, or service, must be open,
documented, and transparent as to the data on which
the decision is based and the criteria applied in judging
the evidence.
• Appropriately applied. Each report, conclusion,
recommendation, and implementation plan must
stipulate accurately and effectively the population to
which the evidence applies, and those using the evidence
must account for the stipulations.
• Patient-focused choices. Decisions on diagnostic or
treatment approaches must take into consideration
not only the appropriateness of the evidence to the
patient, but also patient preferences reasonable to the
circumstances.
• Effective communication and comprehension. Every
application of a diagnostic or treatment product,
procedure, or service must include the most effective
communication possible between patient and caregiver
on the issue, options, choices, and evidence.
• Value driven. Because high-value care means using
the evidence to get the right care at the right time to
the right patient for the right price, each care decision
should be undertaken with transparency as to value,
including price and evidence of effectiveness.
• Accountability and assessment. Accountabilty,
performance assessment, and reporting should be a
core feature in the design strategy for every point of
service in a healthcare system.
EvID
ENcE
2� Institute of Medicine
• Continuous learning. Learning is a continuous and
natural feature of each patient and caregiver healthcare
experience, and evidence development should be an
integral aspect of the delivery of care, not just an activity
performed elsewhere by others.
Learning Healthcare System Concepts 2�
6 cApAcITyExpandingtheefforttolearnwhatworks
Achieving the value we should expect from the health
care we receive depends on how much we know about
the relative effectiveness of various interventions. In
2007, Learning What Works Best was developed for the IOM
Roundtable Working Group on Sustainable Capacity as an
overview of some of the primary issues and strategies for
stronger evidence development. A critical hallmark of the past
half century of progress in medicine has been the development
of many alternatives for diagnosing and treating many of the
illnesses and injuries that occur among Americans. Yet, of
the nation’s more than $2 trillion annual health expenditure,
less than 0.1 percent is invested in assessing the comparative
effectiveness of available interventions, and, of this, the
funds specifically appropriated by Congress for comparative
effectiveness research are just a small fraction. The result is
that far too little is known about the relative advantages and
disadvantages of different diagnostic and treatment options.
A 1994 Congressional Office of Technology Assessment
report noted that longstanding estimates suggest only about
10-20 percent of medical procedures have been formally
evaluated for safety and efficacy. Today, with expanded
medical options and low capacity for assessment, the gap is
likely larger.
Concern about the limited evidence on which to base
reasonable decisions in health care has prompted a number
of proposals on ways to improve the process. Elements of the
concern focus both on the need for substantially expanded
capacity to conduct the necessary research, and on the need
for better and more consistent priority setting, standards,
methods, and coordination of the work that is done. Several
26 Institute of Medicine
COMPARAtIvE EFFECtIvENESS RESEARCH ENtERPRISE MODElS
ACtIvIty FEDERAl INDEPENDENt BOARD HyBRID
Reference model NIH or AHRQ Federal Reserve Board NAS (IOM/TRB)
Priority setting Agency*/HHS Board Governing Board/Staff Governing Board/
ISO**
Budget allocation
Agency/HHS Board Governing Board/Staff Governing
Board/ISO
Study selection Agency Governing Board/Staff ISO
Design/methods Agency Agencies/
IOM-NAS Agencies/ISO
Agency designation Agency Governing Board/Staff Governing
Board/ISO
Studymanagement Agency Agencies Agencies
Study conduct Agency/field Agencies/Field Agencies/Field
Study certification Agency Governing Board/Staff ISO
Study conclusions Agency Governing Board/Staff ISO
Dissemination Agency Governing Board/Staff ISO
Advantages Builds on current Independent Independent
Builds on current
Disadvantages Politically vulnerable No established credibility Other missions
of ISO
Linked to one agency Duplicate capacity
* Some proposals suggest creating an agency-associated but privately operated FFRDC (Federally Funded Research and Development Center) to give the work quasi-insulated status.**ISO: Independent Scientific Organization (e.g., Institute of Medicine, on the model of the NAS Transportation Research Board)
Learning Healthcare System Concepts 2�
proposals have offered different approaches to creating
and managing a stepped-up program of comparative
effectiveness research. In Learning What Works Best, the
primary models were summarized, and the principal current
options can be grouped into three categories, as in the table
at left—one placing primary responsibility in a federal health
agency, one placing it primarily in an independent entity
outside any cabinet department, and a hybrid version using
a federal agency and an established independent scientific
organization.
More recently (2008), the IOM Committee on the
Identification and Evaluation of Highly Effective Clinical
Services, expressed concern over what it characterized as the
fragmented approach and inconsistent standards used in
assessing even the limited research base that exists, and called
for the establishment by the Secretary of Health and Human
Services of a dedicated advisory and program capacity for
the conduct of systematic reviews of the evidence.
Various groups looking at related issues have identified
similar touchstone principles to help guide policy choices
among the various possible approaches to creating a national
entity and capacity to steward and advance comparative
effectiveness research. Core elements include:
• Scientific credibility—ability to gain the trust and
confidence of the public, the scientific community, and
the other stakeholders involved.
• Political independence—well insulated from the
political processes that interests from all perspectives
will seek to leverage.
• Stakeholder neutrality—ability to engage with
all stakeholders—patients, providers, employers,
manufacturers, and insurers—in an independent,
even-handed fashion.
COMPARAtIvE EFFECtIvENESS RESEARCH ENtERPRISE MODElS
ACtIvIty FEDERAl INDEPENDENt BOARD HyBRID
Reference model NIH or AHRQ Federal Reserve Board NAS (IOM/TRB)
Priority setting Agency*/HHS Board Governing Board/Staff Governing Board/
ISO**
Budget allocation
Agency/HHS Board Governing Board/Staff Governing
Board/ISO
Study selection Agency Governing Board/Staff ISO
Design/methods Agency Agencies/
IOM-NAS Agencies/ISO
Agency designation Agency Governing Board/Staff Governing
Board/ISO
Studymanagement Agency Agencies Agencies
Study conduct Agency/field Agencies/Field Agencies/Field
Study certification Agency Governing Board/Staff ISO
Study conclusions Agency Governing Board/Staff ISO
Dissemination Agency Governing Board/Staff ISO
Advantages Builds on current Independent Independent
Builds on current
Disadvantages Politically vulnerable No established credibility Other missions
of ISO
Linked to one agency Duplicate capacity
cApA
cITy
2� Institute of Medicine
• Participatory governance—opportunity for relevant
stakeholders to engage, as appropriate, in priority
setting and agenda setting, while safeguarding the
scientific integrity.
• Investigator integrity—management and conduct
of the research processes, and the determination and
validation of research results completely insulated from
outside influence.
• Agenda flexibility—decision and program processes
with the flexibility to respond quickly to emerging
issues and changing circumstances.
• Infrastructure efficiency—use where possible existing
capacity for the establishment of scientific standards
and for the management and conduct of studies.
• Transparency of processes and results—open access to
the data on which determinations are based and clarity
as to the processes and tools used in their evaluation.
Groups examining the need for the expanded research
activity have estimated increased annual funding needs
for such research ranging from several hundred million
to several billion dollars yearly. Although these may seem
startlingly large amounts, in the context of the $2 trillion
spent on health care, and the potential efficiencies that
might be gained, if only 1 percent (well below the R&D
levels targeted in most industries) of the nation’s healthcare
bill were devoted to understanding the effectiveness of the
care purchased, the total would come to approximately $20
billion annually.
Learning Healthcare System Concepts 2�
7 RESEARchRedesigningtheclinicaleffectivenessresearchparadigm
Information demand is colliding with shortages of
money and time to effect fundamental improvements
in the way evidence is produced on the effectiveness of
diagnosis and treatment interventions. This was the focus
of the Roundtable’s workshop Redesigning the Clinical
Effectiveness Research Paradigm. With the historic advantages
of the randomized controlled trial blunted by time, cost, and
focus, innovative study designs and mathematics, applied to
routine administrative and clinical data are combining to
forge clinical effectiveness research integral to patient care.
Redesigning the clinical Effectiveness Research paradigm
• Address current limitations in applicability of research results• Counter inefficiencies in timeliness, costs, and volume• Define a more strategic use to the clinical experimental model• Provide stimulus to new research designs, tools, and analytics• Encourage innovation in clinical effectiveness research conduct• Promote effectiveness research as a routine part of practice• Improve access and use of clinical data as a knowledge resource• Foster the transformational research potential of information technology• Engage patients as full partners in the learning culture• Build toward continuous learning in all aspects of care
�0 Institute of Medicine
Presenters ranging from clinicians to computer scientists
discussed issues related to certain core, recurrent themes.
• Address current limitations in applicability of research
results. Because clinical conditions and interventions
may be complex, lengthy, and vary in circumstances,
there are different implications for the evidence needed,
the study designs, and the ways lessons are applied: the
internal and external validity challenge. In particular,
given our aging population, often people have multiple
conditions—comorbidities—yet study designs generally
focus on people with just one condition, limiting
their applicability. While our assessment of candidate
interventions is primarily through studies early in
product development, the opportunity for discovery
extends throughout the life cycle of an intervention—
development, approval, coverage, and the full period of
implementation.
• Counter inefficiencies in timeliness, costs, and volume.
Much of current clinical effectiveness research has
inherent limits and inefficiencies related to time,
cost, and volume. Small studies may have insufficient
reliability or follow-up. Large experimental studies may
be expensive and lengthy, but have limited applicability
to practice circumstances. Studies sponsored by product
manufacturers have to overcome perceived conflicts and
may not be fully used. Each incremental unit of research
time and money may bring greater confidence, but also
carries greater opportunity costs. There is a strong need
for more systematic approaches to better defining how,
when, for whom, and in what setting an intervention is
best used.
• Define a more strategic use of the clinical experimental
model. Just as there are limits and challenges to
observational data, there are limits to the use of
experimental data. Challenges related to the scope of
Learning Healthcare System Concepts �1
inferences possible, to discrepancies in the ability to
detect near-term vs. long-term events, to the timeliness
of our insights and our ability to keep pace with
changes in technology and procedures, all must be
managed. Part of the strategy challenge is choosing the
right tool at the right time. For the future of clinical
effectiveness research, the important issues relate not
to whether randomized experimental studies are better
than observational studies, or vice versa, but to what’s
right for the circumstances, and how the capacity can
be systematically improved.
• Provide stimulus to new research designs, tools, and
analytics. An exciting part of the advancement process
has been the development of new tools and resources
that may quicken the pace of our learning and add
real value by helping to better target, tailor, and refine
approaches. Use of innovative research designs, statistical
techniques, genomic information, and probability and
other models may accelerate the timeliness and level of
research insights. Some interesting approaches using
modeling for virtual intervention studies may hold
prospects for revolutionary change in certain clinical
outcomes research.
• Encourage innovation in clinical effectiveness research
conduct. The kinds of “safe harbor” opportunities
that exist in various fields for developing and testing
innovative methodologies for addressing complex
problems are rarely found in clinical research. Initiative
is needed for the research community to challenge and
assess its approaches—a sort of meta-experimental
strategy—including those related to analyzing large data
sets, in order to learn about the purposes best served by
different approaches.
• Promote effectiveness research as a routine part
of practice. Taking full advantage of each clinical
experience is the theoretical goal of a learning healthcare
RESEA
Rch
�2 Institute of Medicine
system. But for the theory to move closer to the practice,
tools and incentives are needed for the caregiver to
participate. One starting point is anchoring the focus
of clinical effectiveness research planning and priority
setting at the point of service—the patient-provider
interface—as the source of attention, guidance, and
involvement in defining the key questions to engage.
The work with patient registries by many specialty
groups is an indication of the promise in this respect,
but additional emphasis on using practice settings
to evaluate interventions is necessary, especially in
anticipation of the access and use of the technology that
opens new evaluation possibilities.
• Improve access and use of clinical data as a knowledge
resource. With the development of bigger and more
numerous clinical datasets, the potential exists for larger
scale data mining and analysis to develop and explore
new insights into the effectiveness and optimal use of
interventions. Taking advantage of these prospects
requires improvements in data-sharing arrangements
and platform compatibilities, addressing issues related
to real and perceived barriers from interpretation of
privacy and patient protection rules, enhanced access
for secondary analysis to federally sponsored clinical
data (e.g., Medicare, Part D pharmaceutical, clinical
trials), the necessary expertise, and stronger capacity to
use clinical data for postmarket surveillance.
• Foster the transformational research potential of
information technology. Broad application and linkage
of electronic health records holds the potential to
foster movement toward real-time clinical effectiveness
research that can generate vastly enhanced insights into
the performance of interventions, caregivers, institutions,
and systems—and how they vary by patient needs
and circumstances. Capturing that potential requires
Learning Healthcare System Concepts ��
working to better understand and foster the progress
possible, through full application of electronic health
records, development and application of standards that
facilitate interoperability, development of new search
strategies for data mining, and investment of patients
and caregivers as key supporters in learning.
• Engage patients as full partners in the learning culture.
With the impact of the information age growing daily,
access to up-to-date information by both caregiver and
patient changes the state of play in several ways. The
patient sometimes has greater time and motivation to
access relevant information than the caregiver, and a
sharing partnership is to the advantage of both. Taking
full advantage of clinical records, even with blinded
information, requires a strong level of understanding
and support for the work and its importance to
improving the quality of health care. This support may
be the most important element in the development of
the learning enterprise. In addition, the more patients
understand and communicate with their caregivers
about the evolving nature of evidence, the less disruptive
will be the frequency and amplitude of public response
to research results that find themselves prematurely,
or without appropriate interpretative guidance, in the
headlines and short-term consciousness of Americans.
• Build toward continuous learning in all aspects
of care. This foundational principle of a learning
healthcare system will depend on system and culture
change in each element of the care process with the
potential to promote interest, activity, and involvement
in the knowledge and evidence development process,
from health professions education to care delivery and
payment.
RESEA
Rch
�� Institute of Medicine
The multifaceted, practice-oriented approach to clinical
effectiveness research discussed at the workshop is
represented in this figure as a continuum in which evidence
is continuously produced by a blend of experimental studies
with patient assignment (clinical trials); modeling, statistical,
and observational studies without patient assignment; and
monitored clinical experience. The ratio of the various
approaches varies with the nature of the intervention, as
does the weight given to the available studies. This enhanced
flexibility and range of research resources is facilitated by the
development of innovative design and analytic tools, and by
the growing potential of electronic health records to allow
much broader structure access to the results of the clinical
experience.
EvIDENcE DEvELOpMENT IN ThE LEARNINg hEALThcARE SySTEM
Recorded clinical outcomes
Models and nonexperimental studies
Experimental studies
Market EntrySafety & Efficacy Safety & Effectiveness
Contribution to Body of Evidence
Periodic Systematic Reviews of the Evidence
Learning Healthcare System Concepts ��
8 DATACreatingaclinicaldatautilityforcontinuousimprovement
The quality, availability, and utility of the evidence to
guide healthcare decisions depend entirely on the
quality, availability, and utility of the data and analysis
to produce the evidence. Important information about the
results of different diagnostic and treatment interventions is
collected in multiple forms by many institutions for different
reasons and audiences—providers, patients, insurers,
manufacturers, health researchers, and public agencies.
Medical care data represent a vital resource for improving
insight and action for more effective treatment. With the
increasing potential of technical capacity for aggregation
and sharing of data while ensuring confidentiality, the
prospects are at hand for powerful and unprecedented
tools to determine the circumstances under which medical
interventions work best, and for whom. Although there
are various challenges to the use of such data—coding
discrepancies, platform incompatibilities, patient protection
tools—practical approaches can be developed for most. The
most significant challenge may be the barrier to data access
and the restrictions in treating clinical outcome data as a
proprietary commodity.
In February 2008, the IOM workshop Clinical Data as the
Basic Staple of Health Learning: Creating and Protecting a
Public Good was convened to facilitate the identification and
exploration of issues and strategies to engage in advancing
the access and use of clinical data for new knowledge and
health improvement. In addition to underscoring the paucity
of translational insights emerging from the vast and growing
body of clinical data, workshop participants discussed the
range of relevant technical, economic, legal, and policy
issues. Several common themes were referenced as priorities
in points made throughout the meeting.
�6 Institute of Medicine
Advancing the clinical Data Utility
• Develop and foster the basic principles of clinical data
stewardship. The starting point for expanded access
and use of clinical data for knowledge development is
agreement on some of the fundamental notions to guide
the activities for all individuals and organizations with
responsibility for managing clinical data. Consensus on
approaches to such issues as data structure, standards,
reporting requirements, quality assurance, timeliness,
de-identification or security measures, access and use
procedures all will determine the pace and nature of
evidence development.
• Provide incentives for real-time use of clinical data in
evidence development. Current barriers to the real-time
use of clinical data for new knowledge—ranging from
regulatory and commercial issues to cost and quality
issues—require a dedicated program of activities,
• Develop and foster the basic principles of clinical data stewardship
• Provide incentives for real-time use of clinical data in evidence development
• Guarantee transparency to the patient when data are applied for research
• Correct the market failure for expanding electronic health records
• Center patients in the learning process through personal records and portals
• Foster coordinated EHR user organization evidence development work
• Develop the business case for expanded data sharing in a distributed network
• Ensure that publicly funded data are used for the public benefit.• Foster initiatives on broader semantic strategies for data mining.• Engage and inform the public in evidence development
strategies
Learning Healthcare System Concepts ��
incentives, and strategies that will improve the methods
and approaches, their testing and demonstration,
the cooperative decision making on priorities and
programs, and the collective approach to regulatory
barriers.
• Guarantee transparency to the patient when data are
applied for research. Patient acceptance is key to use of
clinical data for knowledge development, and patient
engagement and control are key to the acceptance.
Clarity to individual patients on the structure, risks, and
benefits of access to data for knowledge development
is important. Confidence and accountability may be
enhanced through notification and audit transparent to
patients when and by whom their information has been
accessed for knowledge development.
• Correct the market failure for expanding electronic
health records. Currently, market incentives are
inadequate to the necessary expansion of use of
electronic health records. Demand by providers or
patients is not sufficient to counter the expense to small
organizations, competing platforms and asynchronous
reporting requirements work against their utility for
broad quality and outcome determinations, and even
the larger payers—apart from government—do not
possess the critical mass necessary to drive broader
scale applicability and complementarity. It will likely
take a deeper, more directed and coordinated strategy
involving Medicare leadership to foster the necessary
changes.
• Center patients in the learning process through
personal records and portals. Patient demand could
be strategically instrumental in spreading the progress
of the availability of electronic health records for
improving patient care and knowledge development.
Such demand will depend upon much greater patient
DA
TA
�� Institute of Medicine
access to, comfort level with, and regular use of
programs that allow either the maintenance of personal
electronic health records or access through a dedicated
portal to their provider-maintained electronic medical
record. As developing consumer-oriented products give
patients and consumers more active roles in managing
personal clinical information, they may demonstrate
value in the speed and ease of personal access to the
information, better accommodate patient preference
in care, and foster a partnership spirit conducive to the
broader EHR application.
• Foster coordinated EHR user organization evidence
development work. A vehicle to enhance collaboration
among larger EHR users of different vendors may
accelerate the emergence of more standardized
agreements and approaches to integrating and sharing
data across multiple platforms, common query
strategies, virtual data warehousing rules and strategies,
relational standards, and engagement of ways to reduce
misperceptions on regulatory compliance issues.
• Develop the business case for expanded data sharing
in a distributed network. Demonstrate the prospect for
cost savings or avoidance from facilitated feedback to
providers on quality and outcomes; quick, continuous
improvement information; and improved management,
coordination, and assessment of patient care.
• Ensure that publicly funded data are used for the public
benefit. Federal and state funds that support medical
care, as well as support insights into medical care
through clinical research grant funding, are the source
of substantial clinical data that are not yet effectively
applied to the generation of new knowledge.
• Foster initiatives on broader semantic strategies for
data mining. Platform incompatibilities for clinical
data clearly represent substantial rate-limiting factors
Learning Healthcare System Concepts ��
in the spread of electronic health records and their use
for knowledge development. Yet there may be strategies
using alternative semantic approaches for mining
clinical data for health insights, and which warrant
dedicated cooperative efforts to develop and apply
them.
• Engage and inform the public in evidence development
strategies. There is a need to generate a base of support
for and shared emphasis on developing a healthcare
ecosystem in which all stakeholders play a contributory
role. Ultimately, the public will determine the broad
acceptance and applicability of clinical data for
knowledge development, underscoring the importance
of keeping the public closely involved and informed on
all relevant activities to use clinical data to generate new
knowledge.
DA
TA
Learning Healthcare System Concepts �1
9 EXpEcTATIONSResponsibilitiesandexpectationsforparticipants
A continuously learning and improving healthcare
system requires the active engagement of all its
participants, conveying certain expectations for
the roles of each. Growing out of the Roundtable’s Sectoral
Strategies Working Group discussion of core principles for
evidence interpretation and use (see section 5), following is a
summary of suggestions on the expectations for the roles of
each individual and institution involved in the development,
interpretation, and use of the evidence underpinning each
product, procedure, or service in healthcare diagnosis and
treatment.
• Caregivers and health care organizations. For
each healthcare decision, draw where possible on
the assessment and recommendation of a trusted
independent source; know the strength of the evidence
and the attendant risks of relevant clinical options;
know what the evidence says about the appropriateness
of various options for the circumstance at hand; explain
the nature and strength of the evidence to the patient,
and the relative merits of alternatives; seek and consider
patient preferences reasonable to the circumstances; be
mindful of the cost to the patient and to the healthcare
system of alternative interventions; have a strong
scientific rationale if recommending an intervention
insufficiently supported by available evidence specific to
the condition; know the services provided by all medical
caregivers to the patient; and participate in formal
evaluations, including systemwide interventions, to
address gaps in knowledge about the care they provide.
�2 Institute of Medicine
• Patients (or designees). Each patient, or their designee,
should expect the information they need, and in turn,
should be expected to ask questions, express concerns,
and discuss healthcare preferences; achieve a reasonable
understanding of the rationale for a recommended
healthcare intervention, how it applies to the circum-
stance, the nature and strength of the supporting
evidence, and the nature of risk or uncertainty; agree to
close monitoring and documentation when opting for
an intervention in need of additional evidence of
effectiveness; and commit to following through on the
care agreed upon. Patients should also make clear to
their caregivers their expectation that information
about their care and its outcomes should be used to
assess the usefulness of the treatment they receive,
including participation in systemwide intervention
evaluation.
• Healthcare product companies. For each new healthcare
product or service, scrupulously follow established
standards in the development and testing processes;
provide full and transparent documentation of the
testing, safety, and effectiveness of the intervention; if
circumstances preclude customary assessment, offer
full documentation of biologic and clinical plausibility;
meet both the letter and the spirit of requirements in
place to ensure and promote the appropriate use of any
healthcare product or service; represent the products
or services to patients and clinicians in a manner
consistent with the level and findings of the evidence
on benefits and risks; and work with users to arrange
effective tracking and reporting, appropriate to the
nature of the intervention, on the results of its use in
the patient population.
• Researchers. In the design, execution, and reporting
of each study on the effectiveness of a healthcare
Learning Healthcare System Concepts ��
intervention, design a study protocol that reflects,
to the extent practical, the circumstances in which
an intervention is expected to be used; specify
methodological limits, trade-offs in study design, and
possible biases in play; ensure and describe participant
consent, safety, and privacy, including compliance with
all related requirements; describe fully the data and
analytic criteria for their assessment; reveal decision
rules used in drawing conclusions, and, as possible
with appropriate safeguards, make data available as
a resource to others; report results in a form that is
objective and faithful to the data, identifies the nature
of any uncertainties, and allows for accurate translation
and communication.
• Systematic assessors and communicators of evidence.
In formulating each healthcare finding or guidance on
choice of interventions to manage a particular condition,
describe fully and accurately the targeted condition
(nature, impact, course); the proposed diagnostic or
treatment intervention (nature, mode of application,
level of invasiveness, possible side-effects); the nature
of the candidate population (characteristics, size); the
likely follow-on events anticipated from use of the
intervention (other diagnosis or treatments triggered,
duration of impact, longer-term consequences); the
nature and effectiveness of available alternatives; the
studies reviewed (cite, type, duration, size, outcomes,
quality, limitations) and reasons for studies excluded;
the extent to which the studies, individually and as
a body, reflect generally accepted principles for the
production and reporting of evidence; the metrics used
in determining the strength of the evidence; and how
analytic components determined guidance (strength
and quality of the evidence, seriousness of the condition,
effectiveness and consequences of the intervention, and
EXpEcTA
TION
S
�� Institute of Medicine
available alternatives to the intervention). Descriptions
should also include the sources of support for the
work, the consideration of relevant information from
major stakeholders, the clinical and methodological
experts in the assessment activity, and the nature of the
consultative and review processes. Recommendations
should offer clear, succinct, and informative context
for conversations between patients and caregivers;
characterize uncertainties and risks (from condition
or intervention) in terms meaningful for informed
decision making; and underscore issues to clarify as
evidence evolves.
• Regulators. In determining the conditions for allowing
use of a diagnostic or treatment intervention for a
particular condition, maintain evidence standards
that give priority to safeguarding the public and
that provide incentives for the growth of insights on
safety and effectiveness; employ decision rules and
processes that are transparent; be explicit about the
review requirements at the outset and apply them
consistently across interventions of similar potential
impact and risk; ensure that review requirements and
processes yield reliable information on the proposed
intervention’s likely advantages relative to available
alternatives; take into account the anticipated life cycle
for an intervention, so that information gathered in the
premarket period for regulatory approval is structured
to support decision making for delivery and payment
as well; foster innovative approaches to the review of
interventions whose elements are sufficiently similar
to existing interventions as to provide an informative
experience base; tailor the review processes, as
appropriate, for expedited handling of interventions
with the potential for a breakthrough health impact or
significant cost savings; place a premium on monitoring
Learning Healthcare System Concepts ��
protocols for interventions slated for expedited phased
introduction; provide consistent follow-up on post-
approval monitoring requirements; engage in regular
communication efforts with providers and the public
to place into context and improve understanding about
the nature of risks and benefits, the evolving nature of
evidence, and the importance of wide involvement in
the learning process.
• Payers and purchasers. Organizations responsible for
assessing a body of evidence to determine the allowable
payments for healthcare diagnostic or treatment
products, procedures, or services for a particular
condition are expected to encourage continuous
improvement in value delivered; draw where possible
on the assessments and recommendations of trusted
independent sources; ensure appropriate balance of
participants with clinical and methodological skills; be
explicit about the methods and results of analyses on the
safety and effectiveness of the healthcare intervention;
be explicit about cost analyses, and the role of cost-
effectiveness determinations in the decision process;
describe the participants in the work, the process, and
the decision criteria; consider having cost estimates
validated independently by an entity without economic
incentives. EXpEcTA
TION
S
Learning Healthcare System Concepts ��
10 cOMMITMENTSLeadershipcommitmentstoimprovevalueinhealthcare
As a stakeholder leadership forum, the IOM Round-
table on Evidence-Based Medicine provides a venue
not only to highlight and discuss key issues of common
interest, but also to engage in collaborative efforts that reach
out within and across sectors. In 2007, led by Roundtable
representatives, participants from the different stakeholder
communities—providers, patients and consumers, employers
and employees, healthcare organizations, insurers, healthcare
product companies, regulators, information technology
developers, and evaluators—formed working groups to
review the opportunities for each sector to foster progress,
as well as to identify priorities for cross-sector collaboration.
The products of these sectoral working groups were then
discussed at a July 2007 multisector IOM Roundtable
workshop, Leadership Commitments to Improve Value in
Healthcare: Finding Common Ground, and published as part
of the workshop proceedings. Broad common themes of
the discussions included the importance of: building trust
and collaboration among participants; fostering agreement
on what constitutes value in health care; improving public
acceptability of the concepts and terms related to evidence-
based medicine; making the public and policy case for a
stronger focus on evidence identifying the priority evidence
development analyses needed; working to improve the level,
quality, and efficiency of evicence; clarifying and promoting
transparency in health care and related decision processes;
establishing common principles for evidence interpretation
and use; improving engagement of the full life cycle in the
assessment of health interventions; focusing on front-line
providers as change agents; fostering development of a
neutral and trusted intermediary for evidence development;
�� Institute of Medicine
establishing the capacity and incentives for better evidence
development; and accelerating the advances in health
information technology.
Pegging off these general themes, sectoral participants
identified the following projects as priorities for collaborative
work under Roundtable auspices.
• Development of a priority assessment inventory.
Termed a “national problem list” by meeting participants,
this is a multisector collaborative effort to develop
criteria and a list of the diagnostic and treatment
interventions that might be viewed as particularly
important for the development of comparative
effectiveness studies. The list will serve as a means of
illustrating and prompting discussion on the key
evidence gaps and on the design, support, and execution
of the studies needed.
• Pursue agreement on the value proposition. Identify
key concepts and elements to be considered in assessing
and characterizing value from health care, setting the
stage for discussions on approaches to assessing those
elements and applying to add perspective and inform
decision making (see section 4). An IOM workshop,
Making Better Choices: Getting the Value We Deserve
from Health Care, is scheduled for November 2008.
• Identify common principles for evidence interpretation
and use. Identify the core principles underpinning
activities in interpretation and use of evidence, as
background for discussion of the implications and
of the ways the principles might be applied in the
development of a framework adaptive to different
circumstances related either to the evidence base or the
condition of interest (see section 5).
Learning Healthcare System Concepts ��
• Foster cooperative data sharing. Several issues are
important in this regard: platform compatibilities,
standards, economic incentives and disincentives, the
regulatory and privacy environment. HIPAA issues are
being addressed by an IOM Committee expected to
issue its report and recommendations in 2008, including
those related to the use of clinical data for knowledge
development. The Roundtable’s February 2008 meeting,
Clinical Data as the Basic Staple of Healthcare Learning:
Creating and Protecting a Public Good, addressed a
number of the other issues related to sound data
stewardship (see section 8). And collaborative work has
been sponsored by the Roundtable on mining electronic
health records for postmarket surveillance and clinical
safety and effectiveness insights.
• Pursue a public communication initiative on evidence-
based medicine. Use the Roundtable membership’s
collective communication expertise to explore
improving terminology and advancing public awareness
on the nature and importance of evidence in medical
care, the key needs, and the centrality of patient and
provider communication around the state of
the evolving evidence for individual treatment
choices. The Roundtable’s Evidence Communication
Collaborative has a working group actively working on
a communication initiative proposal.
• Support progress on a trusted intermediary for
evidence promotion. The Roundtable’s Sustainable
Capacity working group oversaw the development of
a comprehensive Issue Brief, framing the issues and
options under discussion related to enhancement of the
national capacity to develop, evaluate, organize, validate,
and disseminate information on the comparative
effectiveness of health interventions (see section 6).
Technical assistance and related information is provided
cOM
MITM
ENTS
�0 Institute of Medicine
on an ongoing basis to the various policy discussions of
the issue.
• Identify the potential from best practices in the use of
evidence. It is important to assess and underscore the
best practices in evidence development and application,
including consideration of ongoing methods of
identifying and disseminating those best practices. A
working group is underway to characterize the potential
returns from implementing certain established best
practices.
• Enlist front-line healthcare providers more effectively.
Charge the sectoral working group on providers with
proposing approaches to convening a coalition of
provider groups, perhaps under Roundtable auspices,
to consider sustained, coordinated work on health
professions education, testing, credentialing, and
practice setting tools and structure to improve focus,
accessibility, use, and generation by providers of the
best evidence. A Roundtable collaborative of providers
is being formed to engage this issue.
Each of these activities is now underway, or in the planning
stage, as collaborative ventures of Roundtable members and
other sectoral participants recruited to the process.
Learning Healthcare System Concepts �1
LOOkINg fORwARD
Previewed throughout this Annual Report are various
activities on the emerging Roundtable agenda: workshops
on key issues and concepts of central interest and concern to
Roundtable members; and collaboratives for which a neutral
venue is advantageous to facilitate information sharing
and cooperative initiative to improve the development and
application of evidence for clinical decision making.
2008 Workshops
• Clinical Data as a Basic Staple of Health Learning.
Creating and Protecting a Public Good. To assess the
potential of emerging data capacity to serve as a
compass for continuous improvement in health care,
and to explore some of the technical, economic, social,
and legal challenges to improving the access and use of
clinical data for evidence development.
• Engineering a Learning Healthcare System: A Look at
the Future. To consider, in a collaboration between the
National Academy of Engineering and the Institute of
Medicine, how to harness the tools and experience of
systems engineering and operations research to improve
the efficiency, quality, and outcomes in health care.
• Learning What Works: Infrastructure and Clinical
Priorities for Learning Which Care Is Best. To engage and
characterize the nature and magnitude of infrastructure
improvements to meet the need for better and more
timely evidence for sound clinical decision making.
• Making Better Choices: Getting the Value We Deserve
from Health Care. To conduct an examination of the
needs and approaches to a better understanding and
measurement of value in health care, and its potential
role in guiding healthcare decisions.
�2 Institute of Medicine
2008 Innovation Collaboratives
• Evidence Communication Innovation Collaborative: To
engage participants from multiple sectors in developing
and implementing a communications strategy that
helps foster better dialogue between patients and
providers on the nature, importance, and appropriate
use of clinical evidence to inform and improve health
care choices.
• Best Practices Innovation Collaborative: To bring together
representatives of health professional organizations
for discussion and cooperative action to identify and
improve implementation of best practices in evidence
development and application, including work to assess
the health and economic returns.
• Electronic Health Record User Innovation Collaborative:
To provide a venue for healthcare delivery organizations
using electronic health records to identify common
issues and interests, and to explore collaborative projects
that can facilitate their contributions to advance evident
development and application—and the frontiers of the
learning healthcare system.
• Effectiveness Research Innovation Collaborative: To
engage leaders of the research community—from
public, private, and academic sectors—in work to
improve research methods, identify priorities, and
stimulate activity in comparative effectiveness research
that can generate insights on which interventions work
best for whom, under which circumstances.
Learning Healthcare System Concepts ��
ROUNDTABLE cONTRIBUTORSListed here are individuals who have served as members,
sponsors, advisors, planning committee members, authors,
presenters, formal respondents, or reviewers for Roundtable
activities referenced in this update. We thank them for their
valuable contributions, as well as all others who have attended
and participated in Roundtable-sponsored meetings.
Madhulika Agarwal, Dept VA
John Agos, sanofi-aventis
Rakesh Agrawal, Microsoft
Kate Ahlport, Burroughs Wellcome
Chuck Alston, Manning Selvage & Lee
Stuart Altman, Brandeis University
Steve Altschuler, Chldrns Hosp Phil
David Altshuler, Mass Gen Hospital
Pat Anderson, Stryker
David Atkins, Dept VA
Wade Aubry, Hlth Technology Center
Jerry Avorn, Brighm Wmn’s Hosp
Peter Bach, CMS
Mara Baer, Blue Cross Blue Shield
Bruce Bagley, Amer Assn Fam Phys
Susan Bales, Frameworks Institute
Andy Barbash, Apractis Sol, LLC
Michael Barbouche, Univ Wisc
Annette Bar-Cohen, Nt Brst Ca Coaltn
Jack Bates, VHA
Ray Baxter, Kaiser Permanente
Rachel Behrman, FDA
Karen Bell, CMS
Abdul Bengali, Mayo Clinic
Mark Benton, NC HHS
Marc Berger, Merck & Co., Inc.
Donald Berry, MD Anderson Cncr Ctr
Donald Berwick, Inst Hlthcr Imprvmt
David Blumenthal, Mass Gen Hosp
Ashley Boam, FDA
Carmella Bocchino, AHIP
Kenneth Boff, GA Inst Tech
Marilyn Sue Bogner, Ins Std Hum Err
Queta Bond, Burroughs Wellcome
Catherine Bonuccelli, AstraZeneca
William Bornstein, Emory University
Adam Bosworth, Google, Inc.
Marc Boutin, National Health Council
Jeff Brady, AHRQ
David Brailer, Hlth Evolution Partners
Miles Braun, FDA
David Brennan, AstraZeneca
Jennifer Bright, Mental Hlth America
Sarah Bronko, IOM
Sarah Jane Brown, IOM
Lynda Bryant-Comstock, GSK
Rebecca Burkholder, Natl Cnsmrs Lg
Helen Burstin, Natl Quality Forum
Sam Butler, Epic Systems Corporation
Kathleen Buto, Johnson & Johnson
Robert Califf, Duke Univ Med Ctr
Bruce Nedrow Calonge, CO Pub Hlth
Tanisha Carino, Avalere Health, LLC
Gunnar Carlsson, Stanford University
Kristin Carman, Amer Inst for Res
Margaret Cary, Dept VA
Samantha Chao, IOM
Michael Chase, Kaiser Permanente
Michael Chernew, Harvard Med Schl
Michaele Christian, NCI / NIH
Carolyn Clancy, AHRQ
David Classen, First Consulting Group
Gina Clemons, Dept VA
Amanda Cline, IOM
�� Institute of Medicine
Simon Cohn, Kaiser Permanente
Dale Compton, Purdue University
Paul Conlon, Trinity Health
James Cooper, US House of Reps
Denis Cortese, Mayo Clinic
Peter Covitz, National Cancer Institute
Molly Coye, Health Technology Center
Robert Crane, Kaiser Permanente
William Crown, i3 Innovus
Francesca Cunningham, Dept VA
Gerald Dal Pan, FDA
Helen Darling, Natl Bus Grp on Hlth
Terry Davis, LA State Univ
Nancy Derr, FDA
Don Detmer, AMIA
Amy Deutschendorf, Johns Hop Hosp
Carol Diamond, Markle Foundation
Louis Diamond, Thomson Hlthcr, Inc
Kay Dickersin, Johns Hop Univ
Efrem Dlugacz, Johnson & Johnson
Steve Downs, RWJF
Benjamin Druss, Emory University
Joyce Dubow, AARP
Elizabeth DuPre, Amer Entrprs Inst
Mary Durham, Kaiser Permanente
Carl Dvorak, Epic Systems Corp
David Eddy, Kaiser Permanente
Dee Edington, University of Michigan
Earnest Edwards, formerly Alcoa
Jonathan Einbinder, Prtnrs HlthCr
Alexander Eremia, MedStar Hlth, Inc.
Lynn Etheredge, George Wash Univ
Judy Faulkner, Epic Systems Corp
Nancy Featherstone, AstraZeneca
Mark Fendrick, University of Michigan
Jorge Ferrer, Dept VA
Harvey Fineberg, IOM
Brian Firth, Johnson & Johnson
Elliott Fisher, Dartmouth University
Patricia Flatley Brennan, Univ Wisc
Lee Fleisher, Univ Penn
David Flum, University of Washington
Christopher Forrest, Epic Sys Crp
Joseph Francis, Dept VA
Kathleen Frisbee, Dept VA
Steven Galson, FDA
Robert Galvin, General Electric
Alan Garber, Stanford University
Geoff Garin, Hart Research Associates
Annetine Gelijns, Columbia University
Kim Gilchrist, AstraZeneca LP
Michael Gillam, Microsoft
Marge Ginsburg, Sacrmto Hlthcr Dec
Ron Goetzel, Thomson Hlthcr, Inc
Veronica Goff, Natl Bus Grp on Hlth
Marthe Gold, City College NY
Janlori Goldmann, Columbia Univ
Don Goldmann, Inst Hthcr Imprvmt
David Goldstein, Duke University
Elizabeth Goldstein, CMA
Steven Goodman, Johns Hop Univ
Gail Graham, Dept VA
Sheldon Greenfield, Univ CA, Irvine
Edward Greissing, sanofi-aventis
Thomas Gross, FDA
Jeffrey Grossman, Univ Wisc
Jerome Grossman, Harvard University
Jessie Gruman, Cntr Advncmnt Hlth
James Guest, Consumers Union
Gordon Guyatt, McMaster University
Cecily Hall, Microsoft
George Halvorson, Kaiser Permanente
Bruce Hamory, Geisinger Hlth Sys
Martin Harris, Cleveland Clinic
Tracy Harris, IOM
Winifred Hayes, Hayes, Inc.
Crystal Hayling, Blue Shield CA Fndtn
Anthony Hayward, NIH
Robert Hayward, University of Ottawa
Mark Helfand, OR Hlth & Sci Univ
David Helms, AcademyHealth
Carolina Hinestrosa, Nt Brst Ca Cltn
Louis Hochheiser, Humana, Inc.
Peter Honig, Merck & Co., Inc.
Learning Healthcare System Concepts ��
Carmen Hooker Odom, Mlbnk Mmrl
Julianne Howell, CMS
Courtney Hudson, EmergingMed
Kathy Hudson, Johns Hop Univ
John Iglehart, Health Affairs
Karen Ignagni, AHIP
Belinda Ireland, BJC Healthcare
Telba Irony, FDA
George Isham, HealthPartners, Inc.
Anil Jain, Cleveland Clinic
Brent James, Intrmtn Health Care, Inc
Michael Johns, Emory University
Roger Johns, Johns Hop Schl Med
Michael Johnson, Kaiser Permanente
Peter Juhn, Johnson & Johnson
Douglas Kamerow, RTI International
Jim Karkanias, Microsoft
Sam Karp, CA Hlthcre Fndtn
Stephen Katz, NIAMSD / NIH
Rae-Ellen Kavey, NHLBI / NIH
Marcia Kean, Feinstein Kean Hlthcr
Edward Kelley, AHRQ
Brian Kelly, Accenture
Jeffrey Kelman, CMS
Ann Kempski, Srvc Emp Int Union
Helen Kim, Kosan Biosciences
Joe Kimura, Hrvrd Vngrd /Atrius Hlth
Darrell Kirch, AAMC
Margaret Kirk, Y-ME Ntl Brst Ca Org
Kenneth Kizer, Medsphere
Jim Knutson, Aircraft Gear Corp
Isaac Kohane, Harvard Medical School
Mary Jane Koren, Commonwlth Fund
Ron Krall, GlaxoSmithKline
Barry Kramer, NIH
Missy Krasner, Google, Inc.
Alan Krensky, NIH
Phillip Kroth, Univ New Mexico
Joel Kupersmith, Dept VA
Michael Kussman, Dept VA
Pierre La Chance, Kaiser Permanente
Jeanne Lambrew, George Wash Univ
Carol Lannon, Cincinn Chldrn’s Hosp
Eric Larson, Group Hlth Coop
Richard Larson, Mass Inst Technol
Tara Larson, NC HHS
Michael Lauer, NHLBI / NIH
Cato Laurencin, University of Virginia
William Lawrence, NC HHS
Arthur Levin, CMS
Jeffrey Levin-Scherz, Hrvrd Vanguard
John Lewin, Amer Collge, Cardiology
Louise Liang, Kaiser Permanente
Richard Lilford, Univ Birmngham, UK
Keith Lind, AARP
Chris Lisi, The Glover Park Group
Kathleen Lohr, RTI International
Douglas Lowery-North, Emory Univ
Bryan Luce, United BioSource Corp
Michael Mack, CSANT
Stephen MacMillan, Stryker
Brian Maloney, AstraZeneca
Benjamin Mann, DARPA
Teri Manolio, NHLBI / NIH
JoAnn Manson, Harvard Med Schl
Janet Marchibroda, eHealth Initiative
William Marder, Thomson Hlthcr, Inc
Norman Marks, FDA
Elliott Maxwell, Johns Hop Univ
Mark McClellan, Brookings Institution
Clem McDonald, NLM / NIH
Walter McDonald, Cncl Med Sp Soc
Robert McDonough, Aetna
Terry McGeeney, TransforMED
Barbara McNeil, Harvard Med School
David Meltzer, University of Chicago
Richard Menschel, Charina Endw Fnd
Kate Meyers, Kaiser Permanente
Karen Milgate, CMS
JoAnn Minor, FDA
Deb Mizeur, U.S. House of Reps
Tara Montgomery, Consmr Rpts Hlth
Alan Morris, University of Utah
Ralph Muller, UPenn Health System
�6 Institute of Medicine
Mary O’Neil Mundinger, Columbia U
William Munier, AHRQ
Michael Mustille, Permanente Fedrtn
Elizabeth Nabel, NHLBI / NIH
John Nance, ABC / Phoenix Partners
Mary Naylor, UPenn Schl Nursng
John Neiderhuber, NCI / NIH
Garry Neil, Johnson & Johnson
Eugene Nelson, Drtmth-Htchk Md Cr
Chalapathy Neti, IBM
Peter Neupert, Microsoft
Nicole Newburg-Rinn, Consultant
Len Nichols, New Amer Foundation
Nancy Nielsen, Amer Med Assn
Sharon-Lise Normand, Harvard Med
Leslie Norwalk, CMS
Samuel Nussbaum, Wellpoint, Inc.
Susan Odenthal, Johnson & Johnson
Peggy O’Kane, Ntl Comm Qual Assrn
Elizabeth Olmstead Teisberg, UVA
Alexander Ommaya, Dept VA
Paul O’Neill, Value Capture LLC
Peter Orszag, Congrssnl Budget Office
Eduardo Ortiz, NHLBI / NIH
James Ostell, NLM / NIH
J. Marc Overhage, Regenstrief
Douglas Owens, Stanford University
Andy Page, IOM
Cynthia Palmer, AHRQ
Rachel Passman, IOM
Margie Patlak, Consultant
Avinash Patwardhan, URAC
Steven Paul, Eli Lilly and Company
Ronald Paulus, Geisinger Hlth Sys
Greg Pawlson, NCQA
Lisa Payne-Simon, Blue Shld CA Fnd
Cindy Pearson, Ntl Wmn’s Hlth Ntwk
Steven Pearson, NIH
Gerald Peden, Ind Blue Cross
Steve Pelletier, Pelletier Editorial
Jonathan Perlin, HCA, Inc.
Eric Peterson, Duke Univ Med Schl
Diana Petitti, U So CA Schl Med
Robert Phillips, Robert Graham Cntr
Steve Phurrough, CMS
Richard Platt, Hrvrd Plgm Hlth Care
Michael Porter, Harvard Bus School
Eva Powell, Natl Part Wmn & Fmlies
David Pryor, Ascension Health
Barbra Rabson, MA Hlth Qual Prtnrs
Eric Racine, sanofi-aventis
Gurvaneet Randhawa, AHRQ
Kimberly Rask, Emory University
Sir Michael Rawlins, NICE, UK
Wayne Ray, Vanderbilt University
Rita Redberg, UCSF
Proctor Reid, Natl Acad Engineering
Alison Rein, AcademyHealth
Vicky Rideout, Kaiser Fam Fndtn
Mark Roberts, U Pttsbrgh Med Clg
Alan Rosenberg, Wellpoint, Inc.
Wayne Rosenkrans, AstraZeneca
Murray Ross, Kaiser Permanente
John Rother, AARP
Tim Rothwell, sanofi-aventis
William Rouse, GA Inst Tech
John Rowe, Columbia University
A. John Rush, UT Sthwstrn Med Cntr
Leonard Sacks, FDA
Khaled Saleh, University of Virginia
Judith Salerno, IOM
Lewis Sandy, United HealthCare
Sebastian Schneeweiss, Harvard Med
Deborah Schwab, Blue Shld CA Fndtn
Sanford Schwartz, Univ Pennsylvania
Nina Schwenk, Mayo Clinic
Joe Selby, Kaiser Permanente
Leopold Selker, Evnstn Nthwst Hlthcr
Donna Shalala, University of Miami
Gail Shearer, Consumers Union
Paul Shekelle, Rand
Kristina Shulkin, IOM
Jeffrey Shuren, FDA
Susan Shurin, NHLBI /NIH
Learning Healthcare System Concepts ��
Denise Simons-Morton, NHLBI /NIH
Eve Slater, Pfizer, Inc.
Jean Slutsky, AHRQ
Karen Smith, AstraZeneca
Mark Smith, CA Hlthcr Fndtn
Peter Smith, Duke Univ Med Schl
Thomas Smith, Evnstn Nthwst Hlthcr
Steven Solomon, CDC
Harold Sorenson, UCSD
Steve Soumerai, Harv Med/Harv Pil
Steven Spear, Mass Inst Technology
Stephen P. Spielberg, Dartmouth Coll
Judy Staffa, FDA
William Stead, Vanderbilt University
Donald Steinwachs, Jns Hop Pub Hlth
Andrew Stern, Service Emp Intl Union
Eugene Steuerle, Urban Institute
Mark Stewart, Amer Collg Cardiol
Walter Stewart, Geisinger Hlth Sys
Susanne Stoiber, IOM
Barry Straube, CMS
Nancy Sung, Burroughs Wellcome
Stephen Swensen, Mayo Clinic
Gary Teal, Emory University
Robert Temple, FDA
Jeff Tenner, IBM
Nicolas Terry, St Louis U Schl of Law
Steve Teutsch, Merck & Co., Inc.
James Tien, Univ Miami
Laura Tollen, Kaiser Permanente
David Tollerud, IOM
Sean Tunis, Ctr Med Tech Policy
I. Steven Udvarhelyi, Ind Blue Cross
Steve Vinter, Google, Inc.
Frances Visco, Natl Br Ca Coaltn
Steven Waldren, Amer Acad Fam Phys
Alexander Walker, Harvard University
Elizabeth Walker, Amer Entrprs Inst
Paul Wallace, Kaiser Permanente
Phil Wang, NIMH / NIH
Kerry Weems, CMS
Kenneth Weine, Consumers Union
James Weinstein, Drmth-Hthk Md Ctr
Harlan Weisman, Johnson & Johnson
Kevin Weiss, Northwstrn Med Schl
Jed Weissberg, Kaiser Permanente
Lynda Welage, University of Michigan
William Weldon, Johnson & Johnson
Alan Westin, Privacy Consltng Grp
Ben Wheatley, IOM
John Wilbanks, Science Commons
Marcy Wilder, Hogan & Hartson LLP
Gail Wilensky, Project HOPE
Mark Williams, Emory University
Thomas Winkel, Stryker
Janet Woodcock, FDA
Steve Woolf, VA Cmmnwlth Univ
Jill Yegian, CA Hlthcr Fndtn
Jason Zielonka, Ortho-McNl Janssen
�� Institute of Medicine
Sponsors of the IOM Roundtable on Evidence-Based Medicine
Agency for Healthcare Research and Quality
America’s Health Insurance Plans
AstraZeneca
Blue Shield of California Foundation
Burroughs Wellcome Fund
California HealthCare Foundation
Centers for Medicare and Medicaid Services
Charina Endowment Fund
Commonwealth Fund
Food and Drug Administration
Johnson & Johnson
Gordon and Betty Moore Foundation
National Heart, Lung and Blood Institute
sanofi-aventis
Stryker
Department of Veterans Affairs
Roundtable Staff
Katharine W. Bothner, Administrative Assistant
W. Alexander Goolsby, M.S.P.H., Program Officer
J. Michael McGinnis, M.D., M.P.P., Senior Scholar & Executive Director
LeighAnne M. Olsen, Ph.D., Program Officer
Daniel F. O’Neill, M.B.A., Research Associate
Related IOM Reports To Err is Human, 2000 Crossing the Quality Chasm, 2001 Small Clinical Trials: Issues and Challenges, 2001 Medical Innovation in the Changing Healthcare Marketplace: Conference Summary , 2002 Priority Areas for National Action: Transforming Health Care Quality, 2003 Knowing What Works in Health Care: A Roadmap for the Nation, 2008 The Institute of Medicine serves as adviser to the nation to improve health. Established in 1970 under the charter of the National Academy of Sciences, the IOM provides independent, objective, evidence-based advice to policymakers, health professionals, the private sector, and the public. The mission of the IOM embraces the health of people everywhere.