disclaimer opinions i will voice are not official opinions of nih ... - cogr.pdfstrengthening...
TRANSCRIPT
![Page 1: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/1.jpg)
Strengthening Research Rigor and Reproducibility
Shai D. Silberberg National Institute of Neurological Disorders and Stroke
National Institutes of Health
Disclaimer Opinions I will voice are not official opinions of NIH
Update on NIH efforts to improve rigor
Outcomes of NINDS training workshop
Discuss the role of Institutions
![Page 2: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/2.jpg)
![Page 3: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/3.jpg)
October 2017
November 2017
September 2018
“High-profile cases of exposed malpractice continue to pile up, and surveys of researchers regularly confirm that poor behaviour is shockingly more common than many who promote the values of science might want to accept.”
![Page 4: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/4.jpg)
Nature, 2016; 537: 29-30
Barking up the wrong tree!
Misconduct accounts for the majority of retracted scientific publications
(PNAS, 2012; 109:17028-17033)
“The percentage of scientific articles retracted because of fraud has increased ∼10-fold since 1975.”
0.01%
![Page 5: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/5.jpg)
“Once a man’s understanding has settled on something (….), it draws everything else also to support and agree with it”
The New Organon, 1620
FRANCIS BACON
Unconscious bias
Human Nature
![Page 6: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/6.jpg)
“I used to think that the brain was the most wonderful organ in my body. Then I realized who was telling me this.”
Emo Phillips
licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license
![Page 7: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/7.jpg)
Effect size for studies of FK506 (Tacrolimus) in experimental stroke.
Sena et al., Trends Neurosci 2007; 30: 433-439
The fewer methodological parameters are reported, the greater the apparent efficacy!
![Page 8: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/8.jpg)
Hackam and Redelmeier, JAMA 2006; 14: 1731-1732
Journals: • Cell • Nature • Science • Nature Medicine • Nature Genetics • Nature Immunology • Nature Biotechnology >500 citations
Inadequate reporting is widespread
![Page 9: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/9.jpg)
Publish or perish!
Grant support
Impact factor
Innovation Significance
Novelty
Paper money!
![Page 10: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/10.jpg)
Institution administration
Funders
Journal editors
Reviewers
![Page 11: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/11.jpg)
Research Program Award (R35 Clinical Trial Optional)
RFA-NS-18-032
A more stable funding environment, facilitating the pursuit of longer term research goals
Flexible funding, enabling investigators to pursue research opportunities as they arise, not tied to specific aims
Reduced time spent writing grant applications and managing multiple grant awards, thereby allowing investigators to spend more time conducting and overseeing research
More time for PDs/PIs to mentor junior scientists
![Page 12: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/12.jpg)
Improving the Review of
Manuscripts and Grants
![Page 13: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/13.jpg)
June 20 – 21, 2012 Washington Plaza Hotel
Washington DC
R
Editors
Reviewers Investigators
Funders
![Page 14: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/14.jpg)
Landis, et al., Nature 2012; 490: 187-191
![Page 15: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/15.jpg)
“…we will more systematically ensure that key methodological details are reported, and we will give more space to methods sections. We will examine statistics more closely and encourage authors to be transparent, for example by including their raw data.”
![Page 16: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/16.jpg)
Implementing Rigor and Transparency in NIH & AHRQ Research Grant Applications
Notice Number: NOT-OD-16-011
![Page 17: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/17.jpg)
NIH & AHRQ Announce Upcoming Updates to Application Instructions and Review Criteria for Research Grant Applications
Notice Number: NOT-OD-18-228
“….replacing the term "scientific premise" with "the rigor of the prior research" and adding instruction and review language so that "the rigor of the prior research" is addressed under Significance and Approach”
![Page 18: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/18.jpg)
Education &
Culture
![Page 19: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/19.jpg)
Jointly Sponsored Ruth L. Kirschstein National Research Service Award Institutional Predoctoral Training Program in the Neurosciences (T32)
“The training supported by the JSPTPN must be grounded in principles of rigorous experimental design, an understanding of the critical need for, and proper use of, statistics, and quantitative literacy.”
PAR-17-096
![Page 20: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/20.jpg)
Survey of formal training in the principles of rigorous research at Institutions with neuroscience training grants (n=42)
course (5)
mini course (2)
≥3 lectures (6)
≤2 lectures (11)
none (13)
no response (5)
![Page 21: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/21.jpg)
Building a program from scratch
A free educational resource that is: • comprehensive • modular • adaptable • upgradable
Activation energy
The value of a visionary educational resource to instill the principles of rigorous research
![Page 22: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/22.jpg)
A Visionary Resource for Instilling Fundamental Principles of Rigorous Neuroscience Research Workshop
October 22-23, 2018
https://meetings.ninds.nih.gov/meetings/RigorResource/
Natcher Conference Center NIH Campus Bethesda, MD
![Page 23: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/23.jpg)
Agenda
Keynote address: Prof. Malcolm Macleod
Session 1: What are the fundamental principles of rigorous neuroscience research?
Session 2: How best to instill the principles of rigorous research in traditional settings such as the lab and classroom?
Session 3: How best to instill the principles of rigorous research using non-traditional educational tools
********************************************************
Session 4: Vision of a living educational resource for students, educators, and scientists
Session 5: Recommendations
![Page 24: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/24.jpg)
Major recommendations
Most important principles of rigorous research agreed upon o May take different forms in different settings
Culture change is required at all levels o From undergraduates to senior faculty o Embrace failures
The resource should be targeted to all career stages
Evaluations must include assessment of behavioral change
Community needs to champion efforts and share resources
Institutions need to step up!
![Page 25: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/25.jpg)
CAMARADES: Bringing evidence to translational medicine
Would you fund an institution that did not have a research improvement strategy?
Project planning
Internal Review
Conduct of research Publication
Depositing in institutional repository
Extraction of risks of bias
Quarterly report
Modified from Malcom Macleod
![Page 26: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/26.jpg)
CAMARADES: Bringing evidence to translational medicine
F.F.P.
Preregistration
Research Improvement Strategy nu
mbe
r
Modified from Malcom Macleod
quality
![Page 27: Disclaimer Opinions I will voice are not official opinions of NIH ... - COGR.pdfStrengthening Research Rigor and Reproducibility Shai D. Silberberg National Institute of Neurological](https://reader033.vdocument.in/reader033/viewer/2022050608/5fafc7d51ed3746e7f5d6602/html5/thumbnails/27.jpg)
“Action is the foundational key to all success”
Pablo Picasso