responsible conduct of research data issues karin ellison, biology & society

Responsible Conduct of ResearchResponsible Conduct of Research

Data IssuesData Issues

Karin Ellison, Biology & SocietyKarin Ellison, Biology & Society

What is data?What is data?

• ExamplesExamples– Animal observations or counts– Archival documents– Blood samples– Gels– Instrument read-outs– Interviews– Museum specimens– Photographs, movies, other images– Surveys

Responsible Conduct of ResearchResponsible Conduct of Research

• Issue identificationIssue identification• Relevant standards and normsRelevant standards and norms– University– State– Federal– Disciplinary (Professional Societies)

• Critical thinkingCritical thinking– Options for action– Consequences – Why one choice might be preferred

Data IssuesData Issues

• Should I use/report all of my data?Should I use/report all of my data?

• Who owns research data? Who owns research data?

• Should I share my research data? Should I share my research data?

• How should data be organized?How should data be organized?

• How long should data be kept?How long should data be kept?

• When collaborations end, who should When collaborations end, who should use, publish, and do follow up work with use, publish, and do follow up work with the data?the data?

The Baltimore Case: Key PlayersThe Baltimore Case: Key Players

• David BaltimoreDavid Baltimore– Nobel Laureate, Director, Whitehead

Institute, MIT

• Thereza Imanishi-KariThereza Imanishi-Kari– Assistant Professor, MIT

• Margot O’TooleMargot O’Toole– Postdoctoral fellow in I-K’s lab

Baltimore: The ScienceBaltimore: The Science

• Immune system gene expression in Immune system gene expression in genetically altered micegenetically altered mice

• Weaver D, Reis MH, Albanese C, Weaver D, Reis MH, Albanese C, Costantini F, Baltimore D, and Imanishi-Costantini F, Baltimore D, and Imanishi-Kari T. “Altered Repertoire of Kari T. “Altered Repertoire of Endogenous Immunoglobulin Gene-Endogenous Immunoglobulin Gene-Expression in Transgenic Mice Expression in Transgenic Mice Containing a Rearranged Mu Heavy-Containing a Rearranged Mu Heavy-Chain Gene.” Chain Gene.” CellCell 45, 2 (Apr. 25, 1986): 45, 2 (Apr. 25, 1986): 247-259. 247-259.

Baltimore: Beginnings and EndingsBaltimore: Beginnings and Endings

• BeginningBeginning– O’Toole joined I-K’s lab and assigned a

project building on the Cell paper (June 1, 1985).

• EndingEnding– DHHS appeals panel found that DHHS’s

Office of Research Integrity had failed to prove 19 charges of fraud by preponderance of evidence (June 21, 1996).

Baltimore: The StoryBaltimore: The Story

• 10 10 yearsyears of investigations of investigations• University inquiries (2, MIT, Tufts)University inquiries (2, MIT, Tufts)• NIH scientists investigateNIH scientists investigate• Congressional hearings (multiple)Congressional hearings (multiple)– Including Secret Service analysis of laboratory

records

• Letter in Letter in CellCell with corrections, with corrections, CellCell paper paper retracted, retraction retractedretracted, retraction retracted

• NIH misconduct hearings (multiple)NIH misconduct hearings (multiple)• DHHS misconduct finding appeal (1)DHHS misconduct finding appeal (1)

Baltimore: Ethical IssuesBaltimore: Ethical Issues

• Was there misconduct?Was there misconduct?

• What is an appropriate misconduct What is an appropriate misconduct investigation process? investigation process?

• Did Baltimore behave appropriately? Did Baltimore behave appropriately?

• What are the responsibilities of co-What are the responsibilities of co-authors?authors?

• Women in science Women in science

• Data managementData management

Data: Key QuestionsData: Key Questions

• Do I have to use and report all of my Do I have to use and report all of my data?data?–When can data appropriately be eliminated

from analysis/reporting?–When is dropping selected data falsification

of data?

Baltimore: I-K data analysisBaltimore: I-K data analysis

• O’Toole could not get Bet-1, the antibody O’Toole could not get Bet-1, the antibody used to distinguish between foreign and used to distinguish between foreign and native antibodies in the mice, to work as native antibodies in the mice, to work as reported in the reported in the CellCell paper. paper.

• O’Toole describes meeting with I-K in which I-O’Toole describes meeting with I-K in which I-K analyzed both of their data and eliminated K analyzed both of their data and eliminated outliners in ways O’Toole thought outliners in ways O’Toole thought inappropriate. inappropriate.

• In addition, O’Toole differences between lab In addition, O’Toole differences between lab notes and published figures in the notes and published figures in the CellCell paper. paper.


• I-K appropriate data handling?I-K appropriate data handling?– I-K had explanation for how she treated data. – I-K explained inability to replicate work as

O’Toole’s poor skills.


• Standards?Standards?– Research Misconduct (current definition)

Research misconduct means fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results….

(b) Falsification is manipulating research materials, equipment, or processes, or changing or omitting data or results such that the research is not accurately represented in the research record….

(d) Research misconduct does not include honest error or differences of opinion.

(DHHS, ORI, http://ori.dhhs.gov/misconduct/definition_misconduct.shtml, Accessed 3/26/09)

– Honesty


• How should one respond to this kind of How should one respond to this kind of concern?concern?– O’Toole approached colleagues at Tufts and later

MIT with concerns. • Reasonable choice

– Groups of faculty at each institution reviewed the paper and concluded small errors in paper and professional difference of opinion.• These findings should have ended the matter, but didn’t.

– Other, better options?• Formal misconduct inquiry?


• Who owns research data?Who owns research data?• Should I share my research data?Should I share my research data?– What kind of data?

• Preliminary• Raw• Analyzed data• Results

– Context?• Federally funded research• Human subjects research• Intellectual property (private sponsors, patents)• Classified/military research

Baltimore: Circulating lab notesBaltimore: Circulating lab notes

– Another former I-K lab trainee informed Walter Steward and Ned Feder, NIH scientists and self-appointment fraud watchdogs, of O’Toole’s concerns.

– Steward and Feder approached Margo O’Toole about her concerns.

– O’Toole copied 17 pages of Moema Reis’s laboratory notes and mailed them to Steward and Feder.


• StandardsStandards– Ownership• Federally funded research.• Institution owned data.• I-K as PI was the custodian of the data.


• StandardsStandards– Data sharing

• NIH. Data Sharing Policy and Implementation Guidance (Updated: March 5, 2003).– “In NIH's view, all data should be considered for data

sharing. Data should be made as widely and freely available as possible while safeguarding the privacy of participants, and protecting confidential and proprietary data. NIH requires that investigators share data.”

http://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm

– Share “final research data,” “Recorded factual material commonly accepted in the scientific community as necessary to document and support research findings…”

– Generally computerized datasets, excludes laboratory notebooks

• Openness


– How should one respond to requests for data? • O’Toole could have referred Steward to I-K for

data.• University could have impounded the data when

O’Toole initially raised questions.


• How should I keep my data and How should I keep my data and research materials organized?research materials organized?– Bound notebooks, ink, numbered pages– Computer records–Materials—plates, gels, museum

specimens, etc. – Electronic laboratory notebooks– Languages

Baltimore: I-K’s record keepingBaltimore: I-K’s record keeping

• Did not keep single notebook; had a file with Did not keep single notebook; had a file with loose pages. loose pages.

• Did not do anything systematic with counter Did not do anything systematic with counter tapes immediately. Sometimes organized tapes immediately. Sometimes organized materials months after research. materials months after research.

• Compiled and annotated materials when Compiled and annotated materials when sending to Washington for review. sending to Washington for review.

• Secret Service forensic analysis of lab Secret Service forensic analysis of lab records showed that they were not in records showed that they were not in chronological order.chronological order.

Baltimore: I-K’s record keepingBaltimore: I-K’s record keeping

• StandardsStandards– Lab notebook standards most formalized in

industrial settings and when researchers are concerned with intellectual property

– Carefulness

• Better options? Better options? – Better records might have shortened the

investigations substantially.

Data Management QuestionsData Management Questions

• Should I use/report all of my data?Should I use/report all of my data?

• Who owns research data? Who owns research data?

• Should I share my research results?Should I share my research results?

• How should data be organized?How should data be organized?

• How long should data be kept?How long should data be kept?

• When collaborations end, who should When collaborations end, who should use, publish, and do follow up work with use, publish, and do follow up work with the data?the data?

responsible conduct of research data issues karin ellison, biology & society

Documents