jamelle e. banks, mph sarah glavin, phd eunice kennedy shriver national institute of child health...
TRANSCRIPT
It’s a Small World After All: Describing and
Assessing NIH Funded Research in the Context of
a Scientific FieldJamelle E. Banks, MPH
Sarah Glavin, PhD
Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), NIH
Are we doing a good job selecting investigators, projects, etc.?
Does the work we support meet quality standards?
Is a P01 better than 3 R01s?
If we issued a grant and nobody published, would it make a sound?
Should we support more research in area X v. area Y?
NIH v. the World: What that Comparison Will NOT Allow You
to Do
What is the NIH “market share”?
Who are the other funders in the field?
What do the other funders support, compared with NIH?◦ Overlap/duplication◦ Areas to collaborate and potential collaborators
Where are the gaps in the research?
Where are most of NIH publications along the research continuum (basic, clinical, translational)?
Who is using NIH findings?
NIH v. the World: What that Comparison WILL Allow You to Do
Start with “the world”, and then pull NIH as a subset
OR
Start with NIH, then search for “the world”
NIH v. the World: How do you do it?
Start with “the world”, and then pull NIH as a subset
Advantages:◦ Conceptually cleaner/less biased◦ Less time and resources
Disadvantages:◦ Difficult to characterize an NIH program as such,
because you often do not capture the whole program
◦ Most likely to miss more basic research, so may understate NIH impact
NIH v. the World: How do you do it?
Start with NIH, then expand to “the world” Advantages:
◦ Easier to check whether searching “the world” properly captures NIH publications
◦ Can build criteria for “the world” search to best match NIH program
Disadvantages:◦ More time and resource intensive◦ May be biased in favor of including NIH research
NIH v. the World: How do you do it?
Specialized Cooperative Centers Program in Reproduction and Infertility Research (SCCPRIRs)
16 center grants, $19.6 M in FY 2008 from NICHD
Largely basic science research in infertility and reproduction and some clinical research
Annual publications range from 108 to 170 over past 6 years
An Example: The SCCPRIRs Program
How do SCCPRIRs fit into the world of infertility and reproduction research?◦ Develop list of SCCPRIRs publications from
progress reports and grant acknowledgements – n=121
◦ Use SCCPRIRs keywords to search ISI database for other literature in the field The world’s n ranged from 6,316 to 30,880 articles
(depending on search parameters)
An Example: The SCCPRIRs Program
SCCPRIRs “market share” 0.4% to 2%
Areas of research and journals were fairly similar between SCCPRIRS and the broad literature◦ Broad literature included more publications
specifically focused on animal reproduction
◦ Broad literature emphasized reproductive cancers to a greater degree than SCCPRIRs program
An Example: The SCCPRIRs Program
Journal Name Rank SCCPRIRS (most =1)
Rank in “world” (most =1)
Endocrinology 1 6
Journal of Clinical Endocrinology and Metabolism
2 13
Journal of Biological Chemistry 3 24
Biology of Reproduction 4 3
Molecular Endocrinology 5 31
PNAS 6 26
Molecular and Cellular Endocrinology
7 7
Endocrine 8 49
Fertility and Sterility 9 1
Human Reproduction 10 2
An Example: The SCCPRIRs Program
Other funding sources for reproduction and infertility research◦ SCCPRIRs grants are not co-funded, but SCCPRIRs
researchers received other NIH funding from 7 ICs◦ Non-NIH research projects on reproduction and
infertility were funded by: Private foundations (small numbers each); Japan’s Ministry of Science, Education, and Culture Japan’s Society for the Promotion of Science (quasi-
governmental) Small number of private industry sources
An Example: The SCCPRIRs Program
less than 2.01 2.01--3.0 3.01--7.0 7.01 and higher0
10
20
30
40
50
60
2.59
31.03
49.14
17.24
22.2
26.7
50.31
0.8
Distribution of Articles by Journal Impact Factor, SCCPRIRs and "The World" Literature, 2007
SCCPRIRs The World
Projected v. Actual Citations for SCCPRIRs-Associated Articles,
2007
47%53%
actual > projected projected > actual
Identify collaborators Look for possible overlap and duplication Identify research gaps Help identify impact of research results Develop more realistic expectations and
understanding of NIH’s role in the larger research community
NIH v. the World: How Could You Use These Type of Results?
Questions?
Contact Information:◦ Sarah Glavin – [email protected]
◦ Jamelle Banks – [email protected]
Thank you!