impact observatory of clinical research regarding data sharing karmela krleža-jerić on behalf of...
TRANSCRIPT
IMPACT Observatory of Clinical Research Regarding Data Sharing
Karmela Krleža-JerićOn behalf of IMPACT Observatory
Ottawa Group -IMPACT- Dpt of Research in Biomedicine and Health, University of Split School of Medicine
Karmela Krleža-Jerić
No conflict of interestbut
interest in research &public disclosure of research data
KKJ
Open trial data
• Public health aspect; ultimate user and beneficiary aspects
• International aspect: dialogue, initiatives, actions, standards, IMPACT
• Clinical trial aspect – transition regarding changes in data sharing ;shift of paradigm
KKJ
outline
• Clinical trials• Data sharing• IMPACT • IMPACT Observatory • IMPACT observatory issues
KKJ
IMPACT
IMProving Access to Clinical Trial data
http://ottawagroup.ohri.ca/disclosure.htmlKKJ
CLINICAL TRIAL and RCT
• “clinical trial” is a prospective (research) study evaluating the effects of one or more health-related interventions assigned to human participants.
• RCT (randomized controlled/clinical trial) is using random allocation, determining whether - an intervention is effective or harmful.or- an innovative intervention leads to improved health.
KKJ
5 Open Data
KKJ
Tim Berners-Lee, the inventor of the Web and Linked Data initiator, suggested a 5 star deployment scheme for Open Data.
Available on web under open licence
Structured data (excel instead of image scan of table)
Non-proprietary formats (CSV instead of excel)
Use URI to denote things so peoplecan point on your stuff
Link your datato other data to provide context
to increase reliability of evidence for evidence informed decision
making
Why do we want to open CT data?
KKJ
to speed up knowledge creation and development of better diagnostics
& therapies
Why do we want to open CT data?
KKJ
KKJ
Pooled analysis
data- knowledge-innovation pyramid
New
therapies
Wisdomknow how & what to do
Knowledge - evidence
Information - results
Data – findings
KKJ
And if we do not share data?
KKJ
biased clinician
impact on patients
Lack of information
biased researcher & sponsor
inefficient and irresponsible research
KKJ
IMPACT Observatory story
KKJ
2004
2006
2008
2012
2013
2014
Ottawa Statement
WHO Trial registration standards
PROCTOR Ottawa (CIHR-IRSC)
Environmental Scan of Repositories of Clinical Research Data
Planning Meeting IMPACT Ottawa (British High Commission, University of Ottawa, participants)
IMPACT Observatory NewFelPro Fellowship
kkj-CIHR May 2008
IDEAPROTOCOL
ID#
TRIALID#
ID#
kkj-CIHR May 2008
OTTAWA STATEMENT 1&2
AmendmentsRecruitment statusAdverse eventsInterim analysesEarly stopping (if applicable)
OTTAWA STATEMENT 2
Register 20 WHO items plus
• consent form• details of design• ethics approval• countries & sites• key dates• full protocol• contract/ financial agreements• systematic review
OTTAWA STATEMENT 1(2004)
• ID• Protocol• Results
TRIALID#
DATADATA
DATADATAOTTAWA STATEMENT 3Results Disclosure 1
Content: CONSORT + OS2 itemsSites / CountriesAny independent validation of resultsList of any publications
DATADATA
DATADATA
OTTAWA STATEMENT 3Results Disclosure 2
Individual participant level datasetSystematic review after the trial
DATADATA
Ottawa Statement
Results reporting PROCTOR 2008
• Build research on all knowledge, not just partial
• Current system is suboptimal and inefficient in the Internet era
• Without results reporting trial registration is an unfulfilled promise
• Perform secondary analysis
CMJ 200810.3325/cmj.2008.2.267KKJ
PROCTOR 2008: Public Reporting Of Clinical Trials Outcomes and Results
Design Conduct Analysis
Trial Registry
Results Database
Pro
toco
l Ele
men
ts
Updates
Aggregate Data
Final Update
Results and Findings
Raw
Data- IP
DTrial Data Repository
cross-referenceA
ll Trial D
ocume
ntatio
n
KKJ
Excitement
Hesitation
KKJ
Intellectual property, TRIPS
Resources
CitabilityAvailability
Privacyanonymization
data format standards accuracy quality
Repositories and data exchange standards
Barriers
Academic credit
Prepublication
KKJ
Public Disclosure of Results
Standards (to be developed)
Protocol standards (SPIRIT)
Trial registration (WHO/internatio
nal) standards
Publication reporting standards
(CONSORT)
Better & more efficient trials
KKJ
KKJ
KKJ
IMPACT (2013)(IMProve Access to Clinical Trial data)
• Explore definitions of raw data, meta-data data formats, privacy repositories, access to data for re-analysis and pooled
analysis• Develop
methodologies, standards, guidelines• Inform
operationalisation, implementationhttp://ottawagroup.ohri.ca/disclosure.html
KKJ
Environmental scan of repositories
Objective • Identify and explore essential features and practices of
repositories that • accept clinical trial data including IPD• facilitate their public disclosure
Use/ KT • Inform IMPACT (IMProve Access to Clinical Trial data)
• Raise awareness• Help plan the future
Funded by the COPE grantKKJ
Methodology
KKJ
Selection/inclusion criteria Search Data collection
Repositories that harvest clinical trial raw data with a goal of enabling sharing and public disclosure
InternetRegistries of data repositoriesLiteratureOther sources
Define features-develop headingsExtract data from websites; Interview managersLiterature search (a)repositories(b) methodologies
Data repositories
KKJ
necessary to achieve the full transparency of research open data
store and maintain
data
provide access to
dataarchive
share, reuse
KKJ
Databib n+598re3data n=608
Manual search n=3
Repositories that accept human
datan=93
Repositories excludedDuplicatesInclusion criteria not met
Repositories that accept clinical
researchn=11
Repositories that accept clinical
trialsn=5
Repositories that accept raw data from
clinical trialsn=4+5 (?)
Repositories derived from search
Selected repositories
• Dryad • Figshare• MMMP • Databox• ICPSR • YODA• Res Data Australia • Zenodo• Edinburgh DataShare
KKJ
URL
REPOSITORY N
AME
TYPE OF R
EPOSITORY
GEO-COVERAGE
NETWORKIN
G
ACCEPTABLE
SIZ
E
DEFINIT
ION O
F RAW
DATA
REQUIREM
ENTS TO F
ACILIT
ATE MULT
IPLE
SECONDARY U
SES
TYPE OF R
ESEARCH DATA
CONTROL OF Q
UALITY
PARTNERSHIP
CURATED
UPDATES (ARCHIV
E VS. O
VERWRIT
E)
POLICIE
S
CITABIL
ITY O
F DATA
ETHICS G
UIDELI
NES
DE-IDENTIF
ICATIO
N PRACTIC
ES
PRIVACY
GOVERNANCE MODELS
TYPE OF O
WNERSHIP
COST AND F
INANCIA
L STRUCTURE
ARCHITECTURE
SERVICE L
EVEL AGREEM
ENT (SLA
)
GRAPHICAL
USER INTERFA
CE (GUI)
SOCIAL
NOTIFIC
ATION S
YSTEM
API - A
PPLICATIO
N PROGRAM
MIN
G INTERFA
CE
ONLINE H
ELP
DATA F
ORMATS A
ND STA
NDARDS
WHO C
AN UPLO
AD
SECONDARY USE A
CCESS
SCIENTIF
IC A
REA
SCIENTIFIC AREA
• clinical trials• any research
CITABILITY OF DATA
• DOI• URN• ARK• Handle• PURL• Persistent identifier system HDL• ORCID• cite the repository• none
DATA FORMATS AND STANDARDS
No universal standardsLeave to submitter and journal
Range of standards: • Defined • No specific • No specific fields• Any• As is
Formats: • as is• multiple formats to enable accessibility
WHO CAN UPLOAD
• Anyone• Limited (by application)• Closed (no outside institutions)• Certain scientific area (malignant melanoma)• Only if published
SECONDARY USE ACCESS
Openfree to download & no legal barriers to reuse
Limited /Restrictedupon requestqualified researcher
Conditionsdata use agreement no sharing
Closed responsible organisation only
Combination
PARTNERSHIP
Journals /publishersDryad- BMJFigshare- PLoS, Faculty 1000
ORCID- 37 partners
Funding agencies
Governments
Consortia
PharmaYoda-Medtronic
TYPE OF RESEARCH DATA
• publications • any standard file• images, graphs, video• multimedia & audiovisual• datasets• outcome data• summary data • raw data• individual data elements for each
participant
CONTROL OF QUALITY
Extensively curated
To certain extend
As is
ARCHITECTURE
GUI - rare
Online analysis
Report templates
Social notification
Social commentary
Repositories
• Directories/Registries of research data repositories re3data , Databib
• DataCite • Repositories that take clinical trial data are
General Country/region Institutional Topic
• No universally accepted standardsKKJ
Barriers, opportunities, trends 2013
• Improvement and development needed
KKJ
Directories/Registries of repositories Repositories
Description / categorisation of repositories Features
Search engine Policies
Duplication: same repositories in R3data and Databib
Standards
Coordination btw re3data and Databib No domain repository for CT data
Barriers, opportunities, trends 2015• Number of data repositories increase from 600 in May 2013
to 1275 in June 2015. but
• No new repository hosts clinical trial raw data• No domain repository for clinical trial data
• re3data, DataCite and Databib partner in 2014• re3data & Databib merged in March 2015
• DataCite assigns persistent identifiers -DOI• DataCite manages re3data from May 2015• RDA –Research Data Alliance
KKJ
Barriers, opportunities, trends 2015
• Improvement and development needed
KKJ
Directories/ registries of repositories Repositories
Description / categorisation of repositories
Features
Search engine Policies
Standards
No domain repository for CT data
IMPACT Observatory
IMPACT-IMProving Access to Clinical Trials data; Observatory of ongoing transition in clinical
research regarding data sharing, opportunities and barriers
KKJ
Observatory or natural experiment
• Experiment is not controlled by researcher• No intervention by researcher • Identifies changes/ transition • Points to barriers and facilitators of …• Measures the impact of changes on…• Informs and thus indicates trends
KKJ
IMPACT Observatory
KKJ
explore development in the past
examine ongoing changes of transparency of CT data
measure the impact of changes on clinical research
establish potential future development
Methodology
KKJ
barriersopportunities
gaps
Scoping review
Websites & literature
Surveys & Interviews
Initiatives
Repositories
Indicate trends
Impact transition
Public release of findings
Issues
• no specific MESH terms• no domain repository • no standards for public disclosure and reuse of
data. • Observatory: complex; numerous players
KKJ
Res
earc
her
s
Regulators
Fu
nd
ers
JournalsPublishers
Pharmaceutical IndustryAcademia
Co
nsu
mer
Ad
voca
tes
Med
ia
Data Repositories
PolicyCulture
Infrastructure
Acc
essPreservation
Kn
ow
ho
w
Complexity of observatory
Initiative related issues and how we deal with them
• Initiatives Not much published
KKJ
Internet, contacts
Selected Initiatives
• IOM Report• RECODE Conference- Athens• BioCADDIE- BD2K• RDA• BioCaDDIE Group at RDA• Repository group at RDA
KKJ
Survey
• Monkey survey • Sample: corresponding authors of CTs
published in 2013• Responded: 2250• Limitation:
-low response rate although high number• Peek into some preliminary Results
KKJ
Peek into some preliminary results
• Trial registration: 60 +8% registered/ 22% did not/ 7% did not know
Published studies• Summary results: 21% public….but• Data sets from published studies
last : 58% my computer/43% org web/ 4% sent to journal
earlier: 11% yes/ 35% upon request /53%no
KKJ
Expected results
KKJ
impact the transition
indicate potential trends
identify barriers, gaps and opportunities of opening clinical trial data
identify tendencies of transitions, dynamics and roles of different players
Potential long term outcomes
IMPACT Observatory is established as a longitudinal study/ tool to measure the • Impact of interventions by many players on clinical
trial research How it is done How it is reported How data are shared How secondary analysis is done
• Indicate trends• Establish future developments
KKJ
Who is IMPACT Observatory
• We are - anyone interested• IMPACT Observatory team
• Everything we do depends on collaboration
• Multi nodal network• Network of people/groups of people with good
ideas: OG, PROCTOR, IMPACT, Cochrane, RDA, ECRIN
KKJ
IMPACT Observatory team 2015
KKJ
Karmela Krleža-Jerić, NEWFELPRO, Dpt on Research in Biomedicine and Health, Univ of Split School of Medicine, Split, Croatia
IMPACT Observatory
Thank you
http://ottawagroup.ohri.ca
KKJ
IMPACT Observatory
KKJ
explore development in the past
examine ongoing changes of transparency of PR data
measure the impact of changes on quality/ knowledge management
establish potential future development
KKJ
KKJ
What did you do with datasets of last published trial
KKJ
Saved on my computer Saved in my organisation’s database or website
Sent to a Journal along with the manuscript
Data was saved by sponsor/funding agency
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
Last published trial - what did you do with the datasets/raw data/individual patient data?
PROCTOR
• PROCTOR: Public Reporting Of Clinical Trials Outcomes and Results
• An international meeting held by CIHR in Ottawa in March 2008
• Wide range of constituencies• No consensus building but identified issues and challenges• Identified a need to develop international standards for public
disclosure of trial results
kkj-CIHR May 2008
Participating constituencies
• Clinicians• Researchers• Systematic reviewers• Consumers• Ottawa Group • Policy makers• Trial registries
• Industry• Ethicists• Medical journal
editors• Public Funders• Methodologists• CONSORT• WHO – PAHO
kkj- Oct- 2009
PROCTOR discussion topics
• why• for whom• when • which trials• how much detail• what format• responsibility
• monitoring quality• accountability• website features• funding• who should run websites• barriers
kkj-CIHR May 2008
kkj-CIHR May 2008
Regain public trust
• Provide global open access to information• Reduce unnecessary duplication• Prevent harmful trials to be repeated• Enable monitoring of adherence to ethical
principles• Provide practical benchmark for
accountability
kkj-CIHR May 2008
WHO International Standards
1. Primary Register and Trial ID #
2. Date of Registration in Primary Register
3. Secondary ID#s
4. Source(s) of Monetary or Material Support
5. Primary Sponsor
6. Secondary Sponsor(s)
7. Contact for Public Queries
8. Contact for Scientific Queries
9. Public Title
10. Scientific Title
11. Countries of Recruitment
12. Health Condition(s) or Problem(s)
13. Intervention(s)
14. Key Inclusion & Exclusion Criteria
15. Study Type
16. Date of First Enrollment
17. Target Sample Size
18. Recruitment Status
19. Primary Outcome(s)
20. Key Secondary Outcomes
PROCTOR discussion topics
• why• for whom• when • which trials• how much detail• what format• responsibility
• monitoring quality• accountability• website features• who should/not fund• who should/not run
websites• barriers
kkj- March 2009
What we need to do to achieve the universal results reporting?
Changes culture of researchers regarding
• data sharing• research of existing data
policies of • regulators • funders• journals • pharmaceutical industry (pharma)• academia
data repositories • features• policies• standards for data accepting, curatorship and sharing
KKJ
Speed up knowledge
creation
BENEFITS OF DATA SHARING
32
45
Enable re-analysis, meta and
pooled analysis
Minimize unnecessary duplication of
research
Overcome publication,
and outcome reporting
bias, selective reporting
Use opportunities
offered by the Internet
1
Build on all knowledge,
not just partial 6
EFFICIENT AND RESPONSIBLE RESEARCH
Data repositories
• store and maintain data• provide access to data• archive• share, reuse• necessary to achieve the
full transparency of research; open data;
KKJ
From: from: http://ottawagroup.ohri.ca:
Protocol
Ethics approval
Recruitment
Results
Publication
Unique IDProtocol registration
Public release of protocol
Recording of amendments
Registration of results
Public release of results
Ottawa statement: general time-line for trial registration process (2004)
• (i) to analyse peer review in different scientific areas by integrating quantitative and qualitative research and incorporating recent experimental and computational findings;
• (ii) to evaluate implications of different models of peer review and to explore new incentive structures, rules and measures to improve collaboration in all stages of the peer review process;
• (iii) to involve science stakeholders in data sharing and testing initiatives,
• (iv) to define collaboratively a joint research agenda that points to an evidence-based peer review reform.
KKJ
How
KKJbaseline
ongoing tendencies
Public sharing of findings
potential development in the future (trends)