improving validity and reproducibility of primary care database studies: an online clinical codes...
DESCRIPTION
Poster presented at SAPC 2013 at SAPC conference, Nottingham on 3/07/2013. David A. Springate, Evangelos Kontopantelis, David Reeves Primary Care Research group, Centre for Biostatistics, University of Manchester Publications in healthcare research using electronic medical records (EMR) databases are increasing at an exponential rate. Because of the amount of data available in large EMR databases, it has been suggested that they may be able to provide results of equal validity to randomised controlled trials. EMR studies rely on clinical codes (such as Read codes) to provide standardised and expressive means for medical professionals to record clinical information. The validity of these studies is dependent on (among other things) the validity of the clinical codes that are used to define the population of interest and their disease conditions. Clinical codes should be held to scrutiny in the same way as other methods since if the inclusion/exclusion criteria for a given condition is invalid then so will be the rest of the study. Also, it should be possible to replicate a given study (e.g. in a different EMR database) based on the information provided in the original paper, not possible if the lists of clinical code definitions are not provided. Furthermore, access to historical code-lists allows researchers and clinicians to make incremental improvements to disease and other definitions, building on and avoiding unnecessary replication of previous work There is currently no obligation to publish clinical code lists and no centralised repository to hold them. Consequently, the vast majority of database studies do not publish their clinical codes and as such are impossible to be fully validated or replicated. To illustrate this, we looked at 45 UK case-control EMR database studies indexed on PubMed and found that only five had any record of any clinical codes in their methodology sections. Of these five, only two published code lists in online appendices and only one provided a full set of codes that would allow for proper replication of the study. We have built an online repository where researchers can deposit their clinical codes at the time of publication in a standardised way, as well as download historical code lists from previous studies. We have uploaded a complete set of Read codes for all versions of the Quality and Outcomes Framework and encourage all code lists published by major medical organisations to be deposited. Reproducibility and validity of EMR database studies would be greatly aided if deposition of all clinical codes was a prerequisite for publication of all future database studies. The ability to build on code lists from historical studies during the development of new code lists will also ease a considerable bottleneck in database study design, removing the need for a huge deal of “reinventing of the wheel” each time a new EMR-based study is undertaken.TRANSCRIPT
University of Manchester
Faculty of Life Sciences
PhotoGraphics Unit
The blue instruction columns are for your help
designing and will not print out on your poster
This PowerPoint 2007 and 2010 poster template produces a portrait
style poster. It is initially designed as a two column poster (although
this can be altered – see Modifying instructions below) and includes
the University Logo, Faculty Name and Faculty web address. This
template can be printed to any of the International Standard Paper
sizes (A0,A1,…etc)
The template should help design a well laid out scientific poster and
will hopefully save you time by not having to start from scratch.
The template has all the elements you need to create your poster,
and by simply dragging and dropping the individual poster elements
you can quickly and simply get a well laid out poster.
If you have any questions regarding the templates, or if you want to
get your poster printed by us please contact :
PhotoGraphics Unit on Tel: 0161 275 5258 or
Email: [email protected]
Modifying the Poster Layout
This PowerPoint 2007-10 template has three different column
layouts. To change to one of the different layouts, Right-click your
mouse on the background (not on the columns or title bar) and click
on “Layout” to see the different layout options.
The columns in the provided layouts are fixed and cannot be moved but advanced users can modify
any layout by going to VIEW and then SLIDE MASTER.
Using the placeholders
To add text to this template click inside a placeholder and type in or
paste your text. To move a placeholder, click on it once (to select it),
place your cursor on its frame and your cursor will change to the
symbol of your ‘mouse pointer on a cross’.
You can then, click once and drag it to its new location where you
can resize it as needed.
Additional placeholders for Titles, Text and Images can be found on
the right hand instruction column of this template.
Viewing your poster
To get an idea as to how you poster will look, you can view the
template at the size it will look like when printed. To see what you
poster will look like at A0 view the template at 100%. To view your
poster at A1 view the template at 70% (to zoom in and out go to the
VIEW menu and click on ZOOM to set your preferred magnification).
Removing Template Elements
To remove any of the Sub-Title Bars and text boxes that are already
on the template and that you don’t need, just select them with the
mouse and press the delete key.
Importing Text and Graphics from External
Sources
Text: Paste or type your text into a pre-existing placeholder or drag
in a new placeholder from the left side of the template. Move it
anywhere as needed.
Photos/Images: Drag in a picture placeholder, size it first, click in it
and insert a photo from the menu.
Tables: You can copy and paste a table from an external document
onto this poster template. To adjust the way the text fits within the
cells of a table that has been pasted, right-click on the table, click
FORMAT SHAPE then click on TEXT BOX and change the
INTERNAL MARGIN values to 0.25
Graphs: You should save graphs as image files such as jpeg, in the
programme that you created them in e.g. excel, and then insert
them into the poster using the Photo/Images instruction above. (Don’t copy and paste you graph from the programme e.g. Excel directly into the template as the graph
may not print properly)
Poster Elements
Use the sections provided below to add new elements to your
poster: Drag a placeholder onto the poster area, size it, and click in
it to edit.
Sub-Heading placeholder
Move this sub-heading placeholder onto the poster to add another
sub-heading. Use section headers to separate the sections and
topics on your poster.
Text placeholder
Move this preformatted text placeholder to the poster to add a new
body of text.
Picture placeholder
Move this graphic placeholder onto your poster, size it first, and then
click it to add a picture to the poster.
PhotoGraphics Unit – University of Manchester
1.828 Stopford Building, Oxford Road, Manchester, M13 9PT
Tel: 0161 275 5258
Email: [email protected]
Po
ste
r Te
mp
late
from
ww
w.m
an
ch
este
r.ac.u
k/p
ho
tog
rap
hic
s
Funded by the National Institute for Health Research (NIHR) School for Primary Care Research.
This is a summary of independent research funded by the NIHR. The views expressed are those
of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Background
Centre for Primary Care: www.population-health.manchester.ac.uk/primarycare
Centre for Biostatistics: www.population-health.manchester.ac.uk/biostatistics
For example, in 45 UK PCD case-control studies into diabetes:
Only 5 reported ANY clinical codes in the paper
Only 2 published lists of codes in an online appendix
Only 1 Provided a full set of code lists for the study
So, in 44 out of 45 studies, given the original paper, there is no
way of independently assessing the validity of how diseases,
exposures, treatments or outcomes were defined or of
replicating the research.
1. Shephard et al. Family Practice 2011; 28:4
2. Brookhart et al. Medical Care 2010; 48:6
3. Jordon et al. Family Practice 2004; 21:4.
4. Smeeth et al. Family Practice 2006; 23:5
• All historical code-lists
published on
clinicalcodes.org are freely
downloadable
• The repository will host the
full set of Read codes from
all versions of the Quality
and Outcomes Framework
(QOF) from 2004 – 2012
• Other clinical codes (e.g.
SNOMED, ICD-10) can
also be deposited
• Can also be used to host
database-specific coding
information (e.g. CPRD
consultation types)
• Codes from other
Electronic medical record
database studies can also
be hosted
The repository would be most effective if…
• Deposition of all clinical codes was a prerequisite for publication of all
future PCD studies
• Funding bodies included deposition as a pre-condition for funding
• Authors of historical PCD studies were encouraged to deposit their
codes post-publication
We are working to automate up/download via an API
Challenges
Improving validity and reproducibility
of primary care database studies:
An online clinical codes repository
University of Manchester Centre for Primary Care / Centre for Biostatistics
David A. Springate, Evangelos Kontopantelis, David Reeves and Ivan Olier
Knowledge of the clinical codes used in a PCD study is critical to
determining its validity because…
1. The ways in which populations, diseases, exposures, treatments and
outcomes are defined through code lists can have a large impact on the
results
2. If a code list for a study is unavailable, that study is impossible to
properly validate or replicate
However, there is currently no…
• Obligation on researchers to publish clinical codes lists by journals or
research councils
• Centralised repository to hold archived clinical code lists
This means that the vast majority of published PCD studies do not
publish their clinical codes and so are impossible to properly validate.
The problem
We are developing an online repository where researchers can freely
deposit their lists of clinical codes at the time of publication in a
standardised way, along with links to the original paper and meta-data on
the codes used.
www.ClinicalCodes.org
• Clinical codes can be held to scrutiny and peer-review in the same way
as any other research methods
• Replication of previously published studies (e.g. in different databases) is
facilitated
• Access to historical code-lists allows researchers and clinicians to make
incremental improvements to disease (and other) definitions, building on
and avoiding unnecessary replication of previous work
• Clinical code lists can become a resource for future research in their own
right (e.g. tracking disease definitions through time)
Advantages
www.clinicalcodes.org – online from July 2013
@medcodes – follow us to keep up with updates to the repository
Contact and references
• Large Primary Care Databases (eg CPRD, Qresearch, THIN) are
increasingly used to address a wide range of research questions1
• Much research has been done into establishing the validity of statistical
analysis of PCD data, e.g. the role of confounding variables2, GP
recording quality3 and selection bias4
• BUT PCDs also rely on clinical codes (such as Read codes) to provide
standardised means for medical professionals to record clinical
information. The validity of PCD studies depends upon the validity of the
clinical codes used to define the population of interest, their disease
conditions, exposures, treatments and outcomes
New primary care
database (PCD) studies
are being published at an
exponential rate