dear dr. minhas, · dr rubin minhas bmj associate editor [email protected] in your response please...
TRANSCRIPT
1
Dear Dr. Minhas,
we would like to thank the editors of the BMJ and the external peer reviewers very much for
their interest in our paper. We very much appreciate the thorough evaluation of our
manuscript.
Here are our responses to the questions and criticisms of the editors and reviewers.
We provide point by point replies to all comments made by the BMJ editors and the external
peer reviewers, explaining how we have dealt with them in our manuscript.
According to your suggestion we have enlisted the help of two non-German co-authors, one
from Europe and one from the US. These co-authors contributed tremendously to content and
editing of the manuscript including figures and tables. We re-analyzed the data set following
the advice of a reviewer to calculate a patient/physician ratio, introduced a new table and
changed 2 figures into one table, following the advice of one of our new co-authors, and
corrected some calculation errors. This explains why we have asked for an extension of the
period for re-submission.
In the spirit of transparency and open science our data set of PMS notifications and extraction
sheets are being made available as a supplement to this article.
We are looking forward to the review of our revised manuscript.
With best wishes
Dr. Angela Spelsberg, SM
Am 18.07.2016 um 15:35 schrieb BMJ:
Dear Dr. Spelsberg
Manuscript ID BMJ.2016.033665 entitled "Post-marketing industry funded trials: a contribution to drug safety? Results from a survey of post-marketing industry funded trial notifications to regulatory agencies"
18-Jul-2016
Dear Dr. Spelsberg
# BMJ.2016.033665 entitled "Post-marketing industry funded trials: a contribution to drug safety? Results from a survey of post-marketing industry funded trial notifications to regulatory agencies"
Thank you for sending us your paper. We sent it for external peer review and discussed it at our Editorial meeting. We recognise its potential importance and relevance to general medical readers, but I am afraid that we have not yet been able to reach a final decision on it because several important aspects still need clarifying.
2
The detailed comments are below.
****Most importantly, it is essential that you revise the paper to 1)enlist a non German co-author to help understand and provide the context that would make this accessible to non German audiences, 2) we require the paper to be de-editorialized and de-sensationalized as it is not suitable for publication as research in the BMJ in its current form. ***
We hope that you will be willing and able to revise your paper according to the editors’ comments and the peer review comments, which are included below, so that we will be in a better position to understand your article and decide whether The BMJ is the right journal for it. We are looking forward to reading the revised version and, we hope, reaching a decision.
Please note that resubmitting your manuscript does not guarantee eventual acceptance, and that your resubmission may be sent for further external peer review.
Please don't hesitate to contact me if you wish to discuss this further.
Yours sincerely,
Rubin Minhas
Dr Rubin Minhas
BMJ Associate Editor
In your response please provide point by point replies to all the comments made by The BMJ editors and the external peer reviewers, explaining how you have dealt with them in the article.
EDITORS’ COMMENTS TO AUTHOR:
Rubin Minhas, Rafael Perrera, Tiago Villaneuva, Wim Weber, Georg Goeggla, Jessamy Baggenal, Elisabeth Loder.
Deliberations:
3
-Interesting topic but numerous concern especially the overt bias in this paper. There are many inflammatory statements that should be removed. The authors should be asked to stick closely to the results and not editorialize. Its needs de-sensationalizing too.
We have done our best to remove "inflammatory statements" and to "de-editorialize and to
de-sensationalize" our text.
-More clarity and explanation is needed on several areas, including:
How the system in Germany works.
Germany has a compulsory health insurance system: every person residing in Germany is
insured by an accredited health insurance company (statutory health insurance) with full
coverage of diagnosis and treatment of diseases, rehabilitation, maternity, and also
medical/surgical consequences of accidents. There are 134 legally accredited health insurance
companies. About 90% of the population are covered by one of these health insurance
plans, and about 10 % of the population are insured by one of 52 private health insurance
companies. Germany has the oldest health insurance system in the world, dating back to 1883,
the time of the social insurance laws of Otto von Bismarck. The Umbrella Organization of
Statutory Health Insurance (GKV), mentioned on top of page 4 (unrevised version) of our
paper, comes into play here. The GKV Spitzenverband is the umbrella organization of the 134
legally accredited health insurance companies. And the GKV is one of three organizations
required by law to register PMIFT notifications. A company initiating a PMIFT is obliged by
law to register the respective study by submitting the following items: time, location,
duration, proposed number of patients and physicians, remuneration of physicians per patient,
medication/medical device under study, and since 2009 a study plan plus a final report one
year after termination, to the GKV, the National Association of Statutory Health Insurance
Physicians (KBV), and to the Federal Institute of Drugs and Medicinal Products (BfArM), the
regulatory agency based in Bonn. Physicians organized in the KBV represent the group who
are contributing (participating) most frequently to industry initiated and/ or sponsored
PMIFT. In Germany as in the US and other countries, PMIFT are regularly conducted by
pharmaceutical companies and contract research organizations (CRO). In Germany the
German Medicinal Products Act (AMG) requires notification of all PMIFT to the above
mentioned authorities, while in other countries, e.g. the US only mandated PASS have to be
registered with the FDA.
How likely are companies to do postmarketing studies in Germany vs someplace like the US?
PMIFTs represent major activities of a manufacturer in the life cycle of a licensed
drug/medicinal product. We found that these activities take place not only in Germany but in
many European countries as well as in the US. In Germany we have the special situation that
a company starting any PMIFT is obliged to notify the respective authorities (KBV, GKV,
BfArM). We have found that almost 200 of these studies are initiated each year in the German
market with many of them lasting for several years, and involving patients from many
European countries, but to our knowledge there is no law in these countries requiring all
PMIFT to be registered with a regulatory agency.
4
Phase IV trials or post-marketing studies represent a heterogeneous group of observational
studies. In our paper we are dealing with "Anwendungsbeobachtungen", regulated by the
AMG. The AMG describes "Anwendungsbeobachtungen" (PMIFT) as invaluable to find out
how the drug functions in "real life" and are expected to bring to light rare adverse drug
reactions (ADR). Furthermore, the law states that these studies are by definition not RCTs
but observational studies; they should be integrated into “daily medical practice”. For data
analysis epidemiological methods should be applied. A pharmaceutical company needs not to
ask for permission to do a PMIFT, but once it initiates a PMIFT, it is required by law to
submit a protocol of the study, plus study plan and design since 2009, to the three institutions
mentioned above, namely KBV, GKV, and BfArM. The workload for the individual
physician to participate in a PMIFT is small, because all examinations, questionnaires, and
clinical research forms applied in a PMIFT should not exceed “regular medical practice" as
specified by the AMG. As a result of these rather vague prerequisites, a broad variety of
studies came to light in our investigation on PMIFT registered, roughly one third being post-
authorization safety studies (PASS). PMIFT are done in many European countries and also in
the US. PMIFT are generally registered with a regulatory agency in case the agency requires
their conduct (e.g. PMRs with the FDA in the US and PASS with the EMA in the EU).
Regarding non-mandated PMIFT, the legislation for registration is heterogeneous in Europe.
However, like the AMG in Germany, other European legislations also require PMIFT to be
registered. In Austria, for example, PMIFT need to be registered with the Federal Office for
Safety in Health Care (BASG) if these PMIFT are NIS as defined by the Austrian
Arzneimittelgesetz §2a Abs. 3
(https://www.ris.bka.gv.at/GeltendeFassung.wxe?Abfrage=Bundesnormen&Gesetzesnummer
=10010441), however this is not identical with the AMG. The required data and documents
for registration with the BASG are very similar to the ones requested for notification by the
three legal authorities in Germany (GKV, KBV, and BfArM)
(http://www.basg.gv.at/arzneimittel/faq/nicht-interventionelle-studien/#c5596). An online-
register of the Austrian BASG currently shows a total of 295 ongoing and finalized NIS
conducted by companies and CRO based in Austria and abroad, e.g. Germany, Switzerland,
the UK, etc. (http://www.basg.gv.at/arzneimittel/lifecycle/nis/). In Italy, for example, NIS are
defined by the legislative decree n.211/2003. However, Italian authorities do not register
voluntary PASS. Similarly, in France and the UK, NIS other than mandated PASS are not
registered by the authorities. By contrast, in Spain, non-PASS NIS, so-called post-approval
studies (EPA), require notification/registration with the Agencia Espanola de Medicamentos y
Productos Sanitarios (AEMP) and the Spanish Clinical Studies Registry (REEC). Thus, under
different legal requirements concerning registration, PMIFT are similarly conducted in
Germany as in other European countries.
Might it be the case that studies done in research backwaters are different in many ways from those done elsewhere.
The perception that PMIFT are taking place in "research backwaters" can be explained by our
findings that only very small numbers of these studies are published in peer reviewed
journals. Furthermore, it is very difficult to retrieve PMIFT from medical literature data bases
or clinical trial registries, because of the confusing terminology used for these studies. For
example in the clinicaltrials.gov data base many manufacturers register PMIFT as phase
IV trials, or observational studies, e.g. prospective cohort studies, or other study types.
However, using our notification data, we were surprised to find some of the 558 PMIFT even
in the clinicaltrials.gov registry, labelled as phase IV trial, prospective cohort study,
prospective observational study, and other study type, when in fact they were simple case
5
series without any comparison/control group. It was our aim to shed light on this neglected
but big area of clinical research activities. We agree that many PMIFT activities
represent "research backwaters"; however, the medical community, ethics committees, and
the regulators acknowledge and accept PMIFT as research and as a cornerstone of drug
safety, e.g. as PASS. Furthermore, pharmaceutical companies list PMIFT in their research
budget under the rubric “funding of clinical research”.
What does it mean to "notify" authorities. Do all of these studies actually take place?
To "notify" means that the respective company fulfills the requirement of the AMG, namely
providing the required information (mentioned above) prior to the start of the study. To our
knowledge these PMIFT are actually executed, but it is not certain whether they are all
completed. Since neither the authorities nor we have access to the actual study data, we can
only refer to the notifications and what we could retrieve about the respective PMIFT from
other sources (e.g. clinicaltrial.gov data base, Medline)
What were these studies. There is no sense at all of what the drugs were, what kinds of studies were proposed, what the outcomes were. Were these mandated postmarketing safety studies?
Among the 558 PMIFT registered/notified with the KBV, about one third were PASS. As
explained above, the PMIFT are by definition of the AMG non-interventional, observational
studies, and not RCT. Looking at the study plans, which were present in only 158 of 558
notifications, the designs most often consisted of case series, often wrongfully labelled with
terms such as single-arm cohort studies or prospective observational studies. The 558 PMIFT
cover a large number of drug classes such as biologicals, oncology drugs, anti-psychotic
drugs, antivirals, contrast agents, hormonal therapies, anti-hypertensives, hematology drugs,
and orphan drugs as well as non-prescription drugs, and remedies. In figure 2 of the revised
manuscript the major drug classes covered by the 558 PMIFT are shown.
Those are actually useful in many cases if done well.
We agree that mandated PASS when properly done are useful, but in reality we have seen that
the PASS in our sample were not given special attention by the regulator and were not
performed according to high scientific standards and have not been published. Furthermore,
it is often not clear from the notification documents, whether a PMIFT has actually been
completed. It is also unclear whether the regulatory agencies are aware of ADR occurring in
PMIFT. In one example - the PASS on Pradaxa (PASS 1060.84, registered as starting in
2009 and ending in 2012) in patients with renal impairment - neither the progress of the study,
nor its completion, nor any results were retrievable from PSUR, or PRAC committee
minutes, or the clinicaltrials.gov database, during the time of the study and the following
years until June 2015. In addition, in the study contracts available to us we found strict
confidentiality clauses, obliging the physicians to only report ADR to the manufacturer, not to
the authorities. The Pradaxa example has been included in the discussion section of our paper.
6
The authors must present a balanced account that does not unfairly malign post=marketing studies.
Our purpose is not to "malign" post-marketing studies, which we consider when done
properly of utmost importance for drug safety and public health. It should be kept in mind,
however, that this group of studies has been largely ignored by the scientific community,
since access to study design, methods, and patient level data is exclusively granted to the
sponsors and in case of mandated PASS also to the regulators. Our analyses were only
possible after obtaining the notification documents through court rulings by enacting the
freedom of information laws. This means that a scientific debate on post-marketing studies is
only in the beginning stage. In addition, the confidentiality agreements further jeopardize
scientific merit of PMIFT.
-The title should not be a question. Most people won't know what "industry funded trial notification" is.
Here we partly disagree. As one of the major aims of post-marketing studies is to evaluate
adverse drug reactions, we naturally focus on their potential contribution to drug safety, and
by phrasing our focus on drug safety as a question, we want to stimulate the readers`
curiosity. As reviewer 3 has characterized PMIFT as a “really bad acronym” and as we think
that the title of the manuscript is too long, we decided on the following title: Industry-funded
post-marketing studies: a contribution to drug safety? PMIFT has been replaced by post-
marketing study=PMS throughout the whole manuscript!
-With regard to the money clinicians are paid to do the studies, does this money go to them or is it money they use to actually run the study? If the latter this might not be excessive because it costs a lot to run a study.
The physicians enrolled in a PMIFT do not have any responsibility for running this study, but
sign a contract with the sponsor to provide patient data to this sponsor, who obliges them to
keep strict confidentiality on all aspects of the PMIFT, including ADR. The workload of the
individual physician (clinician) to participate in a PMIFT is small, because all examinations,
tests, questionnaires, and clinical research forms applied in a PMIFT are supposed to not
exceed “regular medical practice activities". The drug prescribed or the tests ordered by the
physician, for example, are paid for by the respective health insurance company of the patient
included in the study, not by the company! The costs for planning, organizing, executing, and
reporting of a PMIFT are borne by the pharmaceutical company. The PMIFT is either run
directly from the office of the company or it has been delegated to a Contract Research
Organization (CRO). Associated costs for participating physicians are limited to the time for
documenting the treatment and potential ADR, usually on a short questionnaire. Thus, the
money paid to the clinician for participating in a PMIFT, means real additional income
without incurring any extra costs.
-There are also some clarifications needed regarding the remuneration definition (per patient? Total?).
The remuneration for the participating clinician has been calculated in two ways: per patient
per PMIFT and per clinician per PMIFT: Per patient per PMIFT means that a clinician, who
participates in a PMIFT registered in the 3-year period, earns on average € 441.00 per
enrolled patient, adding up to total remuneration costs per PMIFT of more than € 500,000 on
average. This means that the overall remuneration costs for the 558 PMIFT registered in the
3-year period amount to more than € 217 million. Per clinician per PMIFT means, that a
7
clinician participating in one PMIFT registered in the 3-year period earns on average €
19,424. See table 2 in the revised manuscript.
The definition of the threshold for rare adverse events need to be in the Intro.
We have put this definition in the Introduction.
Finally, some more information regarding the data available from the other two sources might also be useful or at least should be mentioned in the Discussion.
We found that the numbers of PMIFT notifications registered by the legal authorities (KBV,
GKV, BfArM) during the time period 2008-2010 differed considerably, showing a range of
499-598. This finding is in its self of importance and will be presented in the results section
and discussed in the discussion section. The three legal authorities may never have compared
or cross-checked their PMIFT notification figures, otherwise they should have been surprised
about a difference of 99 notifications between BfArM and GKV and should have tried to
remedy this. We tried to obtain access to original notification documents from all three
authorities, but succeeded only with the KBV. The GKV provided summary listings of
PMIFT to our FOI request voluntarily, withholding physicians´ honoraria as confidential
business information. The BfArM in contrast had to be taken to court but argued that full
access to original documents was too much of a workload, since they would have to screen
each notification page for confidential business information which had to be redacted
accordingly. Only in the lawsuit with the KBV did we prevail, with access to all original
notification documents. Personal data such as physician`s names and addresses and CRO
names were redacted by the KBV. However the KBV gave us the CRO names on a separate
list at a later stage, complying with our request. This is why we used only the 558 PMIFT
notifications of the KBV for all our analyses and results reported in our paper. The number of
notifications to the KBV happens to lie between those reported to BfArM and GKV.
As all the work presented uses descriptive statistics, not much to add. Would require someone with content expertise about them being able to correctly identify any adverse reaction reporting (really <1%?) although given the study sample sizes these are likely to be negligible.
We agree with the words “given the study sample sizes these are likely to be negligible”.
-If these are 'trials', should they not have looked at registration too?
We did this with clinicaltrials.gov, WHO database, and EudraCT, and indeed found that the
big drug manufacturers increasingly register PMIFT as phase IV trials or as other
observational studies in these registries. However, as mentioned before, the terminology used
is ambiguous. For an epidemiologist, a trial is strictly speaking an RCT, going back to
Archibald Cochrane and beyond. However, in the context of drug research the term "trial" is
used for all phases of a drug´s life cycle from phase I-IV. We have decided to abandon the
term “trial” and speak of post-marketing studies (PMS). This will also comply with the
8
criticism of reviewer 3.
- I think in general the reporting needs a lot of work; they might want to recruit a non-German co-author.
We have enlisted the help of two non-German co-authors, Peter Doshi and Christof
Prugger. Peter Doshi is well known to you and is familiar with the U.S. and the U.K.
situation. Christof Prugger is Italian and Austrian and has worked for many years at INSERM
in Paris and at the University of Münster in Germany. He is therefore familiar with the Italian,
Austrian, German, and French health care systems. Since 2015 he is with the Institute of
Public Health of the Charité in Berlin.
REVIEWERS’ COMMENTS TO AUTHOR:
Reviewer: 1
Comments for editor and author
Please note the views expressed are the personal views of the reviewer and not those of any orgnaisation.
The paper has reviewed descriptive statistics of characteristics of post-marketing industry funded trials (PMIFT) in a single European Union (EU) Member State during a three year period 2008 – 2010 based on varying degrees of documentation obtained under freedom of information requests.
While it is acknowledged that obtaining this documentation from various agencies took time (up to 2015), it is considered a major shortcoming that this time period precedes, in particular, the implementation of the European Union pharmacovigilance legislation, Regulation (EU) No. 1235/2010 and Directive 2010/84/EU. This is because this legislation, which become applicable in July 2012, included a strengthened legal basis for post-authorisation studies and principles for their oversight. This new legislation also provided the basis for the development of good pharmacovigilance practice (GVP) including a module on post-authorisation safety studies (module VIII). The issue of lack of publication of results of these studies, a finding of the current paper relating to their appearing in scientific journals, has specifically been strengthened by this GVP. The paper also focusses on PMIFT in the context of ADR reporting – in particular issues with sample size to detect ADRs and reporting to sponsors. The reporting of ADRs in the context of post-authorisation studies is another area that has been subsequently clarified with GVP i.e. Module VI. Furthermore, the EU Directive has implications for national legislation and while reference is made to revisions to the national German legislation in 2009, it is not clear from the paper if there were further
9
revisions in line with the EU Directive. Therefore the major concern is that the paper is of historical interest in documenting practices that may have changed in the meantime in light of changes in legislation.
The time that has elapsed has also coincided with a series of other initiatives aimed specifically at increasing scientific standards, independence and transparency in post-authorisation studies such as ENCePP [Blake KV et al. Pharmacoepidemiology Drug Safety. 2012; 21: 690–696. doi: 10.1002/pds.3281]. As stated in the cited paper, these initiatives were based on the recognised need to improve post-authorisation studies in light of criticism including (i) studies using the same database yielding different results and authors being unable to get to the bottom of the reasons why; (ii) industry-funded studies yielding more favourable results than non-industry-funded studies; and (iii) observational studies being fundamentally flawed as a result of confounding. The efforts for improvement since then also, in addition to those cited above, include the establishment of the Pharmacovigilance Risk Assessment Committee (PRAC) at the European Medicines Agency (EMA) whose mandate places a strong focus on the Committee's role in the design and evaluation of post-authorization studies to ensure they contribute meaningfully to sustainable life-cycle benefit–risk management [Arlett P et al. Nature Reviews Drug Discovery. 2014; 13:395-397.doi:10.1038/nrd3713-c1]. Therefore, again the concern is that the landscape has fundamentally changed since the relevant data on which this study is based were collected.
Of note, the legislation cited above and the various initiatives at an international or EU level mean the findings of the present study based on a single country are limited in their generalisabilty.
Kevin Blake
Confidential Comments to the Editor
(There are no comments.)
We would like to thank Dr. Blake for his interest in our paper and his constructive criticism.
He focuses on mandated PASS whose quality -he argues - should have improved in recent
years "because of the implementation of the EU pharmacovigilance legislation, which became
applicable in July 2012. This legislation included a strengthened legal basis for PASS and
principles of their oversight". We think that our study is not outdated, because the time period
covered extends well beyond 2010 when the new directives came into place. On average the
PMIFT lasted about two years and the PASS, representing about one third of all 558 PMIFT
even longer. As shown with the Pradaxa example, neither PRAC nor PSUR documents could
be identified for the corresponding PASS 1060.84, starting in 2009 and running until 2012. In
2015 we asked the Drug Commission of the German Medical Association to help us with
identifying any pharmacovigilance documents concerning Pradaxa. They found 68 study
related ADR, but could not identify PASS 1060.84 as a source. By June 2015 the
clinicaltrials.gov data base suddenly showed a prolongation of the study period of PASS
1060.84. No corresponding PRAC or PSUR documents could be found. In essence PASS
1060.84 was non-existent in the EU pharmacovigilance documents on Pradaxa. This shows in
10
our opinion that the legislation has changed but that reality is different. Therefore our paper
should be regarded as a stimulus to the current debate on drug safety studies and to the debate
to make data submitted to regulators available for public scrutiny. We agree with reviewer 2,
who states that our "study could also help provide a foundation for additional research into the
PMIFT landscape". Going back to the concerns of reviewer 1, we do not understand, why our
data should not be generalizable, as drug licensing and pharmacovigilance activities have
been harmonized in the EU since 2001. In our view, the standard of pharmacovigilance,
PASS, and PMIFT in Germany may not be worse than the average standard in the EU. It is
also worth noting that Germany makes up about 17% of the population of the EU, with a big
pharmaceutical market.
Reviewer: 2
Comments for editor and author
This is an important effort by researchers to illuminate the practices of post-marketing industry funded trials (PMIFTs). The research in this area has been limited, particularly because many details of PMIFTs have not been publicly accessible. The researchers have diligently uncovered important details about these trials in Germany over a 3-year period, including contracts between companies and physicians for these trials. Their efforts included months of litigation to obtain government documents for the research, which appeared to be a very difficult process. The researchers work raises important questions about government oversight over PMIFTs.With modifications and clarifications, the study could also help provide a foundation for additional research into the PMIFT landscape.
We would like to thank reviewer 2 for his/her interest in our paper, for the appreciation of our
work and for constructive criticism. We hope that "with modifications and clarifications, the
study could also help provide a foundation for additional research into the PMIFT landscape".
We will provide point by point replies to the general and specific comments.
General comments
(1) The descriptive statistics provide a general depiction of these trials. However, it is difficult to make broad findings about the inadequacies of these trials without more information. For example, the mean number of patients can raise questions about the trials, but the number could be appropriate for some studies. Please see specific comments to clarify these findings.
With regard to the intended number of patients per PMIFT we are providing in table 2
(revised manuscript) the mean, median, and the range, and in table 3 (revised manuscript),
which combines figure 1 and 2 of the old manuscript we show that 306 PMIFT intend to
enroll only up to 1000 patients (1-1000), while 135 PMIFT intend to enroll between 1,001-
10,000 patients, and only 21 PMIFT intend to include more than 10, 000 patients. For 96
PMIFT data on intended number of patients are not available.
11
(2) It could be helpful if the authors selected some trials as case studies for the reader. For example, the authors could focus on important examples of prescription drug approvals in which the government required a PMIFT to address unresolved safety questions. The authors could describe the details of the PMIFT and any information about the trials, particularly their results and any follow-up by the government agencies based on the trial results, such as new warnings or restrictions on sales. These examples might provide more helpful context for the authors’ findings.
We fully agree with this suggestion and come back to the above mentioned example on
Pradaxa. We have studied the Pradaxa case in detail and have provided a comprehensive
report in the discussion section of our paper.
(3) The authors should provide more explanation of the government agencies’ oversight and role for these PMIFTs. For example, were the PMIFTs required by the government agencies? How do the government agencies regulate the PMIFTs? The US FDA requires PMIFTs under certain circumstances and regulates the design and conduct of the PMIFTs, including inspections of study results and data. Did the government agencies here take any actions against companies for inadequate PMIFTs?
About one third of the 558 PMIFT were labeled PASS, i.e. studies required or mandated by
the regulatory agency BfArM at the time of the drug`s licensing or thereafter. To our surprise
PASS were not treated differently from other PMIFT, which raises the question whether the
specifications of the law are followed by the companies and how this is controlled for by
regulators. To our knowledge BfArM, KBV, and GKV did not take any action with regard to
inadequate PMIFT notifications. The big difference (n=99) in the number of registered
notifications between the three legal authorities is in our opinion a clear indication that
a systematic control mechanism is lacking. This impression is corroborated by the finding that
quite a number of mandatory items were missing such as information on intended number of
patients, and/or intended number of physicians, or physicians` remunerations. For example in
96 of the 558 PMIFT the intended number of patients was not available.
(4) The contracts prohibiting adverse event disclosure by physicians to the government raises significant concerns. Have the government agencies addressed this issue or taken a position? The authors also note that information about numbers of participating patients and physicians was lacking in one-third of trials, which is a regulatory violation. Have the government agencies addressed this problem at all? Lack of adequate oversight by the government agencies may be a more important study finding than the descriptive statistics themselves.
We fully agree that the contracts signed by the participating physicians, obliging them to
report ADR solely to the company sponsoring the PMIFT are a major finding which to our
knowledge has not been reported before. We also discussed this problem with the Drug
Commission of the German Medical Association, since these contracts are in conflict with the
physicians` professional codes of conduct. Furthermore, many contracts were not only
drafted in German but also in English and often bi-lingual versions were provided. We agree
12
that lack of adequate oversight by the regulatory agency aggravates the hindrance of reporting
and disclosure of ADR.
(5) Did the authors request government documents that might show how the government agencies addressed these violations by companies for trials, such as emails or correspondence?
We tried to obtain original documents from BfArM, but access was denied by court decision.
(Verwaltungsgericht Köln= Administrative Court Cologne). KBV and GKV clearly stated
that they had no interest pursuing any violations of the companies.
(6) Another important theme seems to be the difficulties that the authors faced in obtaining documents from the government agencies about the PMIFTs. A related theme is the lack of publication about trial results, addressed below in the specific comment #12.
We agree that full access to PMIFT data is equally important as access to RCT data.
However, while the latter has been recognised by the scientific community, the debate on
access to PMIFT data is just emerging; we hope that our paper will contribute to this theme.
(7) Given the limitations in drawing strong conclusions from the descriptive statistics, the authors could consider focusing more analysis on the policy and regulatory themes described in comments ##2-5 or the lack of public information or knowledge about these trials described in comment #6.
We are grateful for these suggestions and have included respective paragraphs in the
introduction and discussion sections.
(8) The authors need to acknowledge and explain the limitations in the Discussion section.
We agree and have added a respective paragraph on the limitations of our study at the end of
the discussion section.
Specific Comments
(1) Abstract (line 5) and Background (page 3, line 6): “Current legislation in Europe and the US relies on post-marketing industry funded trials (PMIFT) for drug safety surveillance.” I believe this sentence should be “Regulatory agencies in Europe and the US rely….”
13
We agree and have changed the text as follows: “Regulatory agencies rely on such industry-
funded post-marketing studies (PMS) for drug safety surveillance”.
(2) Background (page 3, line 8-14): I think a citation/reference would be helpful to support this point of the aim of PMIFT.
We agree and provide the following source: German Medicinal Products Act (AMG), section
4, subsection 23, version 2004, Bundesgesetzesblatt I page 2031, July 30, 2004. This is the
above mentioned harmonization of the EU Directive 2001.
(3) Background (page 3, lines 29-49): I understand the points being made by the author but believe the assertions are overly broad. Post-marketing studies that are carefully regulated by a governmental agency, with valid design, close oversight, and good execution, can be scientifically informative. I think more recent literature supporting these points about the extent of inadequate PMIFTs would be helpful.
We have searched Medline on more recent literature on the quality of PASS/PMIFT but
could not find any quantitative or systematic evaluations of the subject. A recent paper by
authors affiliated with EMA considers the changes of EU-Regulations and Directives
in 2010 as “major positive shifts in pharmacovigilance". However, the paper does not provide
convincing evidence that the problems with ADR reporting have been resolved. Source: Borg
JJ et al European Union pharmacovigilance capabilities: potential for the new legislation.
Ther Adv Drug Saf 2015; 6: 120-140.
(4) Methods (page 4, lines 35-42): Additional clarify about the document request would be helpful. What were the “original documents received by these institutions”? Are these documents submitted by the company to the government agency? Does the government agency first request (or require) that the company conduct a PMIFT? What about the government agency documents, such as a government request to submit a PMIFT, any other government correspondence to the company about the PMIFT, any internal documents with the government’s evaluation of the PMIFT?
The only original documents (notifications, contracts, study plans, redacted lists of
participating physicians, etc.) we were able to access were those submitted to the KBV. The
Administrative Court of Cologne having jurisdiction over the BfArM (located in Bonn)
followed the BfArM`s argumentation that full access to their documents on PMIFT would be
too much of a workload because all the documents had to be redacted for data confidentiality
and CBI. The BfArM was ordered to fulfill our information request and provided only
tabulated lists, which are much less informative than the original documents. The GKV
voluntarily provided such tabulated lists and was not taken to court by us. Overall, about one
third of PMIFT were PASS. We had no access to documents other than 558 notifications, of
which 158 had study plans.
(5) Methods (page 4, lines 35-52): Did the authors assess what adverse events were being studied in the trials and the characteristics of trials studying a certain type of adverse event?
14
We wanted to find out whether ADR are reported from PMIFT to the regulators at all and
how this actually takes place. So far we can only report that the regulators do not connect the
PMIFT notifications with ADR reporting.
This information would be very helpful for the authors’ findings, particularly to know if a trial was adequately powered.
Considering that detecting a rare ADR with an incidence of 1 in 10 000 with a power of
95%, sample sizes of at least 30 000 patients would be necessary. In our sample of PMIFT
306 out of 558 have enrolled only up to 1000 patients. See table 3 (revised manuscript).
Did the authors consider assessing types of products? For example, it could be helpful to know the descriptive statistics for classes of drugs, particularly for more serious diseases (such as cancer). Also, were any of these trials for products with expedited approval?
Yes, we have included a list of drug classes. See figure 2 of the revised manuscript. We also
looked at the subset of PMIFT on recently (<two years) approved drugs with respect to
sample size and physicians` remuneration. We saw no difference with respect to the number
of enrolled patients (see table 3 of the revised manuscript) but a larger number of physicians
was intended and remunerations were twice as high as in PMIFT of older drugs.
(6) Methods (page 5, line 34-39): What does it mean that the reports were “anonymously solicited”?
Anonymously solicited reports mean that the BfArM publishes ADR reports from organized
data collection systems (in contrast to spontaneous reporting schemes) i.e. studies including
PMIFT without providing identification of the source study or individual study.
We will add in the text "with no reference to the individual study".
(7) Results (page 6, lines 54-58): Do the authors know why the numbers of trials differed between the three different agencies? Do the authors believe that only 40 trials are missing from their sample (i.e., 598-548)? Could there be other PMIFTs that are not registered with the government agencies?
We do not understand why the law was violated in this way. Obviously, the three authorities
are not doing any cross-checking and this assumption was confirmed during the court trial
against the BfArM. We do not know if any of the three registries is complete.
(8) Results (page 7, line 13): Could the authors provide the exact percentage of notifications that did not include a study plan or protocol? Are the companies required to submit the study plan or protocol with the notification? If so, then the authors should emphasize in the Discussion that this low percentage of reporting study designs or protocols is a serious violation.
Yes, in about 72% of notifications a study plan was missing. In 2008 43 out of 185 PMIFT
notifications, in 2009 40 out of 185 PMIFT notifications, and in 2010 75 out of 188 PMIFT
notifications had a study plan. This adds up to 158, which means that 400 PMIFT
notifications had no study plan. Since study plans were mandatory by 2009, it is remarkable
that still in 2010 only 40% of PMIFT notifications had study plans.
15
(9) Results (page 8, lines 3-17): The authors conclude that the mean number of patients in these trials is rather small. It would be helpful if the authors could provide more information about the mean number of subjects for trials that support product approval as a comparison.
A few examples may illustrate our point: Revlimid (lenalidomide) has been investigated with
two studies involving a total of 2,000 patients, one comparing Revlimid with placebo the
other comparing the drug with a standard treatment of multiple myeloma.
In the PMIFT on Revlimid only 100 patients and 18 physicians were involved.
Pradaxa (dabigatran etexilate) has been investigated with two main studies comparing the
drug with enoxaparin (another anticoagulant) in patients who had undergone a hip or knee
replacement. The first study involved a total of 2,101 patients who had had a knee
replacement operation and the second involved a total of 3,494 patients who had had a hip
replacement. In our first PMIFT, PASS 1060.84 on Pradaxa, only 300 patients with renal
impairment were included. The second PASS 1060.85 involved 2,036 patients. These few
examples show that the number of patients enrolled in a PMIFT is smaller than the respective
number in the RCT. We could easily prolong this list of examples.
(10) Results (page 8, lines 19-35): The authors conclude that the mean number of physicians is a relatively high number. It would be helpful if the authors could provide more information about how many clinicians/investigators are typically involved in trials that support product approval as a comparison.
We have added a few examples (Revlimid, Pradaxa) in the discussion section illustrating our
point. The corresponding data on the RCT were derived from the EMA homepage.
Also, would it be possible for the authors to calculate a physician/patient ratio for each study, and then provide the mean/median figure and range across the trials?
In the results section we added the calculation of a patient/physician ratio for each PMIFT
with a median of 8 patients per physician, a mean of 85 patients per physician, and a range of
1 to 10, 000 patients per physician (PMIFT with contrast agents).
(11) Results (page 8, lines 38-58): The authors conclude that the mean remuneration is high. Could the authors please provide information about how much physicians are typically paid for trials supporting product approval as a comparison?
The information on RCT remuneration to physicians is not available to us.
Could the authors describe what services the physicians are providing for these types of trials, in order to provide additional context?
As mentioned before in the responses to the editors, the role of physicians in PMIFT is limited
and according to contracts the participating physicians are expected to fill in pre-specified
questionnaire(s) and/or ADR forms. The PMIFT is run by the company or a commissioned
CRO. The role of the physician is to select suitable patients, document their data, and forward
them to the company.
16
(12) Results (page 9, lines 10-19): Could the authors please clarify where they searched for the ADRs?
We have searched for ADR in the UAW data base of BfArM, which contains solicited and
spontaneous reports. Unfortunately, these data do not allow the identification of the source
study and therefore we were not able to relate these data to our PMIFT sample.
Did they search in a separate database, in addition to the documents?
Yes, see explanation above!
Also, did the authors search for journal articles through 2015?
We searched Medline from 2008 to 2015 with the following search strategy: non-
interventional post-marketing study OR post-marketing NIS OR post-marketing trial OR
company observational post-marketing study OR industry funded post-marketing trial OR
adverse drug reaction AND (when available) the study ID AND drug name.
This lack of publication could also be an important theme to stress and may further support the point that many of these studies are not advancing scientific knowledge about product safety.
Yes, we agree and this is what we have stressed in our text.
Did the authors calculate when each trial was scheduled to end?
The notification is required to provide a starting and ending date. From this we have
calculated the mean, median, and range of duration (in days) of PMIFTs. Please refer to table
2 of the revised manuscript.
That end date would be useful when considering the lack of publication through 2015.
We have calculated that 21 of the 558 PMIFT (about 4%) are going to run beyond 2015. This
means that the huge majority of PMIFT should have been reported by now, considering that
results need to be made available one year after completion of the study.
(13) Discussion (page 9, lines 41-48): What would the “real study documents” include? Could the authors please clarify why the estimates from the original notification would likely be an underestimation of PMIFT?
The real study documents include the study protocol, contracts, and questionnaires, all patient
level data, data analysis and reporting of results. First of all there is the discrepancy of 40
notifications between the GKV and the KBV. Secondly, the completeness of many
notification items was unsatisfactory. For example in 196 out of 558 PMIFT we could not
calculate the remuneration per physician, which means that we might have underestimated the
true incentives for physicians to participate in these PMIFT. However we have put this
paragraph in a rephrased form under limitations in the discussion section.Fi
17
(14) Discussion (page 10, lines 13-15): The point about spontaneous reporting seems more applicable for adverse event reporting systems and less applicable to randomized controlled trials. Also, I think more recent literature should be referenced.
We tried to find more recent references and double checked with our co-authors but were not
successful. In our opinion this is an indication for a lack of research activities in this field.
(15) Discussion (page 10, lines 35-40): It would be helpful to briefly state here (or in the Results section) the evidence that ADRs with PMIFT “may have been never or only partially reported.” In the Results section, the authors stated that there were no ADRs with the documents.
We will elaborate on this theme by reporting on the Pradaxa example in the discussion
section.
(16) Discussion (page 10, lines 43-47): This sentence appears unnecessary because it simply repeats the points in the prior paragraph.
Yes, we agree and have deleted this sentence.
(17) Discussion (page 11, lines 5-14): I understand the authors’ point, but the sentence seems overly broad in asserting that the PMIFT are of “low quality” because the authors were not able to assess the study design, conduct, or results. The confidentiality clauses in the contracts, though, are an important factor.
We have rephrased this sentence and agree that the confidentiality clauses in the contracts are
of major concern. Figure 3 in the revised manuscript exemplifies this problem and is
discussed in detail in the discussion section.
(18) Discussion (page 11, lines 14-19): The UK data is informative but from the mid-1990s. Could the authors provide more recent literature?
More recent data published in 2010 are also presented. See reference 34 of the revised
manuscript, Cox et al. Br J Clinical Pharmacology 2010; 69:529-34
(19) Discussion (page 11, lines 23-37): Can the authors provide more evidence or argument to support the point that these ADR non-disclosure clauses in physician PMIFT contracts are related to the decrease in ADR reporting?
As mentioned above, literature on ADR reporting is generally sparse. Our concern is the role
of physicians in ADR reporting. Their professional obligations to report ADR are in conflict
with the confidentiality clauses in PMIFT contracts. We think that this association needs to be
investigated by further studies.
Are there enough PMIFT contracts to influence the overall levels of day-to-day reporting by physicians?
We do not know, but as nearly 200 PMIFT are initiated in Germany each year, a figure which
may not seem extraordinarily high, it is of concern that a high number of physicians are
regularly participating in these PMIFT. The majority of these physicians are GPs.
These high numbers of physicians involved could well exert influence on ADR reporting.
18
(20) Discussion (page 10, lines 48-53): What does it mean that the study design and conduct were “exclusively in the realm of the sponsor”? Did the government agencies not have oversight or final approval over the study design?
A company can initiate a PMIFT and decide on the design and conduct of the study. The
company need not ask a regulatory agency for approval of the study. What the law (AMG)
demands is the registration of the PMIFT prior to the start of the study. Until 2013 this
applied also to PASS. Since 2013 a PASS protocol has to be submitted to BfArM or the
PRAC of the EMA for approval; however, this is only obligatory if the PASS was mandated
by BfArM or EMA.
(21) Conclusion (page 13, lines 19-36): The recommendation that PMIFT should be banned entirely seems overly broad. Post-marketing studies that are carefully regulated by a governmental agency, with valid design, close oversight, and good execution, can be scientifically informative.
We agree and have rewritten the conclusion section, focusing much more on the problems
with confidentiality and asking for more transparency. In the interest of drug safety and public
health PMIFT data should be available for scientific evaluation and public scrutiny.
Confidential Comments to the Editor
(There are no comments.)
Reviewer: 3
Comments for editor and author
This is a potentially interesting article from an obviously very unique dataset that emerged as a result of some impressive persistence on behalf of the authors. This could be a potentially very powerful analysis.
We would like to thank the reviewer for his/her thorough evaluation of our paper, for the
appreciation of our work, and for helpful and constructive criticism.
General comments:
-- As someone who does not have a working knowledge of the German pharmaceutical regulatory apparatus, I found myself at times struggling to understand the difference between the KBV, GKV, and BfArM, the significance of the 2009 change in the law, and the basic reporting and post-market surveillance requirements. A brief summary in the introduction might help. Or maybe the authors should consider simplifying the manuscript -- since it ended up that only data from one of the groups made up most of the analysis, maybe cut out the other stuff, which makes for a mildly interesting back story, but doesn't make up the bulk of the descriptive analysis.
We have inserted a paragraph in the introduction section of our paper, explaining very briefly
the respective specifications in the German Medicinal Products Act (AMG). We would like to
19
re-emphasize that the German drug laws have been harmonized with EU legislation and
therefore our data on PMIFT are to a large extent generalizable to the pharmacovigilance
situation in the EU. PMIFT are conducted in many countries but to our knowledge registration
of all PMIFT with three authorities is only mandatory in Germany. Only since 2009 the AMG
asks for a study plan to be submitted with the notification to the three authorities, namely
KBV, BfArM, and GKV.
With the clarifications given in the introduction section of our paper we hope that the
reviewer accepts that we keep the comparison between the three agencies, because the
unexpected discrepancy in the registered numbers is an important finding in its self.
-- As I think I understand it, the PMIFTs (a really bad acronym, by the way) being registered that made up the source data are projections by the pharmaceutical manufacturers, not completed trials, or trials that the companies are required to conduct under German law? How do they differ from the postmarket requirements/commitments imposed by the FDA? How does this intersect with the EMA and its requirements?
We are dealing with original notifications of PMIFT to the authorities prior to their beginning.
It is correct that there is no real study data in the notification. Furthermore the notifications
comprise mandated and non-mandated PMIFT. Since 2012 the EU requires registration of
mandated PASS and these are registered in the ENCePP. To our knowledge FDA has an
internal register for keeping track of mandated pharmacovigilance studies be it RCT or
observational studies. In contrast to the EU, FDA plays a major role in planning and
overseeing PASS. Throughout the manuscript we have replaced PMIFT by industry-funded
post-marketing study (PMS), complying with the comment “PMIFT = a really bad acronym”.
-- A publication rate of <1% is bad...actually so shockingly bad that I don't believe it. Publication bias etc. is well-known, but it is nowhere in the ballpark of <1%. So what's going on? If PMIFTs are just projections, could it be that companies are not actually even starting them? Or maybe it's the fact that nearly all of them are case series?
We consider a publication a paper in a scientific journal, not a summary report published on
the website of a manufacturer or in a data base of a pharmaceutical association. According to
the notification documents to the KBV, periodic updates of physician enrolment occurred
frequently. This is an indication to us that the studies are real. However, we do not know if all
initiated studies were actually completed and if the intended numbers of patients and
physicians were actually reached. Yes, it is our impression that most of the 558 PMIFT we are
dealing with are in fact case series.
It's probably very unlikely that a case series of an approved drug used for its approved indication is publishable. So why are they being done?
Yes, we agree and this is why we are so surprised that so many studies of this type are being
done. And this is why we think our findings should be communicated to stimulate a scientific
debate on PMIFT!
What is the role that PMIFTs are supposed to play in the German pharmaceutical market?
One aspect is that they can be used as marketing instruments (seeding studies). Another aspect
is that they are considered by the AMG and the regulators as a cornerstone of
pharmacovigilance, a claim/aspiration which is not supported by our findings.
20
In the US, postmarket requirements/commitments are specifically requested by the FDA to fill knowledge gaps...but it doesn't appear like PMIFTs in Germany are intended to fill a similar role. What role do they fill?
In the US the FDA is involved in the planning and oversight of PMR and PMC. This was not
true for the EU until 2012. In Germany only since 2013 study protocols of mandated PASS
have to be approved by regulators at the national or EU level. Our sample of 558 PMIFT
represents a mixed bag of mandated PASS, PASS without mandate, and PMIFT initiated by a
sponsor.
Anyway, getting back to the publication issue, if these so-called "studies" are never meant to collect publishable data, criticizing them for not being published is arguably unfair.
The purpose of PMIFT as specified by the AMG is to further knowledge on drug safety.
However, how shall this aim be achieved when strict confidentiality clauses are enforced and
no access to the data is given for independent evaluation? And without publications there can
hardly be a scientific debate.
-- The authors criticize PMIFTs for being too small to detect some rare ADRs, but that's always going to be true, assuming PMIFTs are prospective trials/case series.
We do not agree since in the pharmacoepidemiologic literature there are well known
examples of prospective cohort studies or case control studies assessing, e.g. the risk of OC
use and stroke or pulmonary embolism in young women. The reason why these studies are not
done any longer is that they are time consuming and costly. However, it would be much better
to have fewer studies of high quality than many PMIFT of unclear scientific value.
The way to detect rare ADRs is with retrospective pharmacoepidemiology and pharmacovigilance. So that's not really a fair criticism either.
We do not agree completely. Retrospective designs are not suitable for newly licensed
medicinal products.
-- The physician remuneration data is interesting, but could use more context. What does it mean for a physician to earn 200,000 EUR, for example, for a PMIFT? It does seem like a problematic system to have so much money flowing from industry to physicians in the context of "studies" of apparently nil scientific value that don't end up leading to any useful information.
We fully agree and this is why it is so important to make our findings publicly known.
I wouldn't call it "hush money" (excessive rhetoric alert), but it doesn't look good, that's for sure. Are these arrangements just part of the German medical culture/system? More background about them would be useful in helping readers understand. Are there anti-kickback laws in Germany? How do these not run afoul of them?
We have removed the term "hush money" in the abstract and the conclusion section.
Interestingly, we did not have an anti-kickback and an anti-bribery law for health care
21
professionals in Germany until July 2016!! According to disclosure of payments to physicians
by the Physicians´ Payment Sunshine Act in the US and voluntary disclosure of
pharmaceutical companies in Europe it is quite clear, that physicians are accustomed to
receive large amounts of money and other incentives from the pharmaceutical industry.
Specific comments:
-- Background: what evidence is there that more expensive drugs are more likely in seeding trials than other drugs? Or that more expensive drugs are me-too's? This whole sentence confuses me.
Our statement has been described in the literature by Kessler DA et al in the NEJM 1994 =
reference 20 in the revised manuscript and more recently by Psaty and Rennie in JAMA
2006= reference 19 in the revised manuscript. These findings are well documented and this is
why we think our sentence is clear.
-- The timing is also confusing. Something happened in 2009? But the data are from 2008-2010? Was it purposeful to select data that spanned the law change? Could the authors describe more any changes they saw in the data from pre- to post-legal change?
We wanted to study what impact the law change might have on the quality of the
notifications. The change implicitly implied that the notifications had to be accompanied by a
study plan. However, as mentioned before the adherence to this law change by the
three authorities was remarkably low in 2009 with only a slight improvement in 2010. There
was also no oversight and no law enforcement.
-- The shout-out to the Volkswagen problems was a bit superfluous.
Here we wanted to particularly emphasize the important role of regulators. The VW case
clearly shows that even the best laws can be worthless (for a long time) if there is no law
enforcement and if the role of regulators is ambiguous. Therefore, we consider the article in
Nature also relevant to our theme.
-- Can you "ban" a PMIFT? How would you do so? Through the medical professional societies? The legislature?
As long as the AMG supports PMIFT there is no ban in sight. A change can only be achieved
by the legislature. A first step would be to delete confidentiality clauses from the contracts
between physicians and companies, especially with respect to ADR reporting.
-- The quick nod to "a better way" in the final paragraph was too thin. How would these studies be funded? Who would do them? How would they be set up. It's hard to throw out big ideas like that at the end without any further context.
In our eyes sponsorship and scientific responsibility in PMIFT have to be disentangled. As
suggested for other areas of medical research it must be guaranteed that the study data are not
confidential business information and are not the property of the sponsor. By the same token
publication of results including ADR reporting must be independent from the sponsor. We
have rephrased the conclusion section accordingly.
22
-- It may be in the German professional code of conduct for docs to report all ADRs to the government, but of course no one really does that, so that's a bit of a paper tiger. I didn't understand what role these contracts play -- why do physicians sign them?
Physicians sign these contracts with confidentiality clauses, because a signed contract is a
prerequisite for a physician to participate in the study. A PMIFT is attractive for physicians,
because the offered honoraria are considerable and the workload per patient is rather limited.
Physicians may assume that by participating in PMIFT they contribute to valuable research.
They may also be told by the sponsor that the PMIFT has been approved by an ethics
committee. We are concerned that signing such a contract may undermine the crucial role of
physicians in detecting and reporting ADR.
-- Table 1: What does the page length matter?
With the term “number of pages per notification” we wanted to give the reader an idea about
the volume and variability of the original notification documents. About one tenth of the
notifications consisted of only 1-2 pages. This is an indication that legal requirements have
been ignored which has obviously been accepted by the authorities. In table 2 of the revised
manuscript we have relabeled the term "pages per notification" into "number of pages per
notification".
-- Table 2: What are those numbers in parentheses? Table 2 was a bit opaque. So there was 1 PMIFT that anticipated over 500 physicians and 500-1000 patients? I don't see the relevance of breaking down the information like this.
Table 2 changed to table 4 in the revised manuscript. The numbers in parentheses are
percentages. Yes, there was one PMIFT that intended 500 physicians enrolling only one or
two patients each. In our eyes such a trial represents the "ideal" of a seeding trial. Table 4 of
the revised manuscript further illustrates this point, showing that the majority of these trials
aimed for up to 500 physicians to participate. Table 4 also shows that although mandatory by
law, information on numbers of participating patients and/or physicians was lacking in one
third (n=182(96+86)). During the court trial with the BfArM in Cologne we learned from one
representative of the regulator that BfArM`s officers would not go after missing items and
would not question violations of the notifications.
Figure 3 is similarly complex. What's the point of breaking down the categories like this? Is it to show a trend of some sort? Or lack of a trend?
Figure 3 changed to figure 1 in the revised manuscript. It shows the calculated physicians`
honorarium per PMIFT by different enrolment categories. Figure 1 reveals a lack of a trend
between study size, workload of physicians, and calculated remunerations. Under the
assumption that the workload of physicians per patient is relatively similar and the study
duration does not vary much between study size (patient enrolment categories) it is impressive
that the range of remunerations paid to physicians is so large. In our opinion this indicates that
payments may follow other intentions than reimbursing physicians for invested time efforts.
In addition it may be of interest that missing values for remunerations were particularly
common in smaller studies. We would like to keep figure 1 of the revised manuscript and
23
have provided a more comprehensive figure legend plus a better description and explanation
of figure 1 in the revised results section of our paper.
Confidential Comments to the Editor
(There are no comments.)
**INFORMATION FOR SUBMITTING A REVISION**
Please submit your revised manuscript within one month.
Once you have revised your manuscript, go to https://mc.manuscriptcentral.com/bmj and login to your Author Center. Click on "Manuscripts with Decisions," and then click on "Create a Resubmission" located next to the manuscript number. Then, follow the steps for resubmitting your manuscript.
You may also click the below link to start the resbumission process (or continue the process if you have already started your revision) for your manuscript. If you use the below link you will not be required to login to ScholarOne Manuscripts.
https://mc.manuscriptcentral.com/bmj?URL_MASK=967d09b91d2c40ba9e01022c1997dea6
IMPORTANT: Your original files are available to you when you upload your revised manuscript. Please delete any redundant files before completing the submission.
Please check that you have completed each of the steps below:
1. The following Licence for Publication statement needs to be inserted into the text version of your manuscript:
Licence for Publication
The Corresponding Author has the right to grant on behalf of all authors and does grant on behalf of all authors, an exclusive licence (or non exclusive for government employees) on a worldwide basis to the BMJ Publishing Group Ltd to permit this article (if accepted) to be published
24
in BMJ and any other BMJPGL products and sublicences such use and exploit all subsidiary rights, as set out in our licence (http://group.bmj.com/products/journals/instructions-for-authors/licence-forms).
2. A signed BMJ patient consent form must be submitted. The template form can be found at: http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists/patient-confidentiality/patient-consent-fo
3. A signed BMJ originality of work attestation form must be submitted. The template form can be found at: http://static.www.bmj.com/sites/default/files/attachments/resources/2016/03/Originality_of_Work_Attestation.pdf
4. Each author and all co-authors must individually complete The BMJ declaration of financial interests form before they submit their manuscripts. The form can be found at:https://docs.google.com/a/bmj.com/forms/d/1fUXxlzMqkWgOQRpPRgNXMlgzD1V8RyNHfPD27-LE5OQ/viewform?c=0&w=1