Challenges to Evidence Based Medicine – the need for more and

better research

S Arulkumaran

Professor Emeritus in O&G

St George’s University of London

““A Fresh Look atA Fresh Look at””Interpreting EvidenceInterpreting Evidence

RCTs & Meta AnalysisRCTs & Meta Analysis

Case control studiesCase control studies

Prospective descriptive studiesProspective descriptive studies

Retrospective data analysis/ case Retrospective data analysis/ case controlledcontrolled

Confidential enquiriesConfidential enquiries

Different search mechanisms; selection bias; different ways of Different search mechanisms; selection bias; different ways of analysis > different results with meta analysisanalysis > different results with meta analysis

Evidence Based Medicine

The mention of the words often invites mixed reactions from medical fraternity

There is no return from this

Exclusion and Inclusion criteria should be spelt out

We need to refine the studies (e.g. adequacy of numbers) and conclusions drawn

Background Some specialists believe that they have

already been doing their best for their patients at all times and there is nothing in their practice which needs a change.

It is also argued that there is very little evidence for majority of what we do in medicine and so EBM may not be necessary.

IS THIS TRUE?

Challenging that notion

Is general inpatient obstetrics and gynaecology evidence-based?

A survey of practice with critical review of methodological issues.

Aamir T Khan1, M Nauman Mehr1, Anne-Marie Gaynor2, Malcolm Bowcock3 and Khalid S Khan1 BMC Women's Health 2006, 6:5 

Evidence

The study examined the rates of evidence-supported care provided in an obstetrics-gynaecology unit. (sample of 325 consecutive inpatient admissions)

Conclusion

A significant majority (90%) of obstetric and gynaecological care was found to be supported by substantial research evidence.

Then what are the problems/challenges of EMB in Obstetrics?

Obstetricians in less resourced countries are worried that they will not know how to search for, critically appraise, analyse and implement the available evidence for the benefit of their patients.

Majority of good quality evidence from NICE, RCOG, FIGO, GLOWM is now available free through internet

Challenges need to be appropriate

Some argue that EBM is a cookbook approach to medicine, and so it may not take cognizance of individual patient’s needs and circumstances.

Caesarean delivery at maternal request for a 42 year old woman with 3 miscarriages and 1 failed previous IVF cycle who is at 41+3 weeks now – would you offer it?

Evidence is to guide Mrs Average’s treatment and not the exception as the trials are on Mrs Average

Challenges In some situations, gold standard evidence

may not be available.

The amount of resources needed to conduct large randomised trials to obtain sufficient evidence is often significant

Thus funding sources may ultimately determine which therapies are subjected to review and which are not.

Challenges The quality of individual studies performed to

obtain evidence may vary, which therefore makes it difficult to compare them and apply the results to general population

All the evidences produced may not be made accessible, and this may bias the results/effectiveness of any particular approach or intervention.

Breech delivery

A number of questions were raised following publication of the Term Breech Trial, largely about selection criteria and the conduct of labour.

How many offer assisted VD as a routine?

How many offer assisted VD on request?

Planned caesarean section for term breech deliveryG Justus Hofmeyr1,*, Mary Hannah2, Theresa A Lawrie3

Editorial Group: Cochrane Pregnancy and Childbirth Group Published Online: 22 APR 2003Assessed as up-to-date: 2 AUG 2011

New Questions posed?

It has been suggested that the Term Breech trial, by reflecting conventional ‘expert’ views, sanctioned the conventional dorsal lithotomy position for delivery and thereby missed an opportunity to evaluate labour and delivery in upright positions.

RCOG Greentop Guideline 20b

Which results to consider? More recently, an observational prospective study with an intent-to-treat

analysis conclude that, in units where planned vaginal delivery is a common practice and when strict criteria are met before and during labour, planned vaginal delivery of singleton fetuses in breech presentation at term remains a safe option that can be offered to women.

2526 women with planned vaginal deliveries, 1796 delivered vaginally (71%). The rate of neonatal morbidity or death was considerably lower than the 5% in the Term Breech Trial (1.60%; 95% CI 1.14–2.17), and not significantly different from the planned caesarean section group).

Goffinet F, et al. PREMODA Study Group. Am J Obstet Gynecol 2006;194:1002–11.

Another study from Gerry Visser’s group supports elective CS based on their large population based study

Preterm labour

Use of a tocolytic drug is associated with a prolongation of pregnancy for up to 7 days but with no significant effect on preterm birth and no clear effect on perinatal or neonatal morbidity.

There is no clear evidence that tocolytic drugs improve outcome and therefore it is reasonable not to use them.

However, tocolysis should be considered if the few days gained would be put to good use, such as completing a course of corticosteroids or in utero transfer.

RCOG Greentop guideline 1b

Preterm labour There is insufficient evidence for reaching any

firm conclusions about whether or not maintenance tocolytic therapy following threatened-preterm labour is worthwhile.

Should we prescribe tocolytic therapy until more research data is available for those who stop and restart preterm labour and are too early to deliver and has no evidence of infection?

Is it lack of evidence or no evidence?

Are there sufficient studies?

Prolonged pregnancy In the trial by Hannah ME et al, 3407 women with low-risk

singleton pregnancies at 41 weeks were randomly assigned to IOL within 4 days (with and without cervical ripening) or expectant management until 44 weeks.

Women in the induction group underwent CS significantly less frequently than the women who were expectantly managed (21.2 vs. 24.5%, respectively; P=0.003) due primarily to a lower rate of CS for non reassuring fetal heart rate tracings in the induction group.

Hannah ME et al. Canadian Multicenter Postterm Pregnancy Trial Group. N Engl J Med 1992; 326:1587–92.

Prolonged pregnancy

A subsequent cost–benefit analysis of these data showed that routine IOL at 41 weeks resulted in significantly lower costs than expectant management ($2939 vs. $3132 Canadian, respectively; P<0.05).

Goeree R, Hannah M, Hewson S. Cost-effectiveness of induction of labour versus serial antenatal monitoring in the Canadian Multicentre Postterm Pregnancy Trial. CMAJ 1995; 152:1445–50.

Prolonged pregnancy – when to induce labour?

Despite these data, routine induction at 41 weeks has not been universally accepted.

Criticism has been made regarding biases in the study by Hannah ME et a (which forms bulk of the evidence in systematic reviews) and it has been suggested that analysis by actual treatment received rather than intention to treat indicated a higher risk of caesarean sections.

Menticoglou SM, Hall PF. Routine induction of labour at 41 weeks gestation: nonsensus consensus. BJOG 2002; 109:485–91.

Method of induction may have influenced the CS rates; PG was used in the active group and oxytocin /ARM in the conservative group

Other Concerns It must be remembered that with a given CTG trace, the

clinical actions and decisions vary depending on the overall clinical picture.

A pathological CTG showing fetal tachycardia with atypical variable or late decelerations and reduced variability at 3–4 cm cervical dilatation in a primigravida might warrant a CS, whereas at 7–8 cm it might indicate the need for FBS and an instrumental delivery in the second stage, if this can be

carried out safely.

CANNOT BE RIGID WITH EBM

Other situations – Delivery of Twins

Twins – Need for well-designed RCTs of interventions to reduce preterm birth in women with twin and triplet pregnancy and short cervix.

Conflicting data about circlage for multiple pregnancies.

Need for evidence regarding optimal management when there is EFW discordance of 25% or more, including optimal timing of delivery.

Other situations – Episiotomy for previous 3’rd/ 4’th degree tears

Episiotomy and ¾ degree perineal tear prevention.

Evidence says no role but experienced obstetricians feel episiotomy helps.

How many would do elective episiotomy or would you base your decision on patient’s views/ request.

Maternal request CS

CS at maternal request – Debate continues especially regarding neonatal benefits versus maternal safety issues

+ Long term effects.

Are we taking into account individual circumstances such as high risk maternal conditions, assisted conception treatments, small family size, increasing maternal age etc?

HYPERTENSIVE DISEASE IN PREGNANCY AND THE USE OF LABETALOL

There is one small head-to-head RCT comparison of oral labetalol vs parenteral hydralazine (Walker JJ et al. Postgrad Med 1983) - underpowered to draw any conclusions. 

On the basis of this trial, however, considerable experience has been gained in Yorkshire as part of their successful guideline (Tuffnell DJ. BJOG 2005)

Upon that success - recommendation was made in the 2011 NICE guidelines that oral labetalol be the drug of choice. 

It was listed as the agent of first choice.  The onset of action of oral labetalol is 20-120 minutes, which about the same as methyldopa and intermediate-acting nifedipine

Peter von Dadelszen

Other than the published pharmacodynamics (e.g., drug monographs), the most supportive methyldopa data come from Hamilton M & Kopelman H. BMJ 1963, which show that the time to effect in patients receiving methyldopa and diuretic together i.e. similar to women with pre-eclampsia who are volume constricted.

Currently, in PRE-EMPT Trial - Nifedipine, labetalol and methyldopa are tested in a RCT of women with severe pregnancy hypertension.  The trial is led by Hillary Bracken from Gynuity and recruiting in Nagpur, India.  

OR

for PTD

95%CI

< 16 weeks 7.55 1.80 – 31.65

< 20 weeks 4.20 2.11 – 8.39

> 20 weeks 1.82 1.06 – 2.21

Bacterial vaginosis as a risk factor for preterm delivery: a meta-analysis.

Leitich et al. Am. J. Obstet. Gynecol. 2003; 189:139-147.

18 trials 20,232 patients

The earlier the diagnosis of BV the greater the risk of adverse outcome

Study Number Mean GA at random

Category Outcome Antibiotic group

Control group

OR 95%CI

Carey et al 2000* 1919 19.7 All

Low risk

High risk

< 37

< 35

< 32

< 37

< 37

116 / 953

48 / 953

22 / 953

86 / 852

30 / 101

121 / 966

49 / 966

26 / 966

95 / 857

26 / 109

0.97 (0.74 - 1.27)

0.99 (0.66 – 1.49)

0.85 (0.48 – 1.52)

0.90 (0.66 – 1.23)

1.35 (0.73 – 2.49)

Hauth et al 1995* 258 22.9 High risk < 37 54 / 172 42 / 86 0.48 (0.28 – 0.82)

Joesoef et al 1995 681 20.3 All

All

< 37

< 32

51 / 340

16 / 340

46 / 341

9 / 341

1.13 (0.74 – 1.74)

1.82 (0.79 – 4.18)

Kekki et al 2001 375 13.5 Low risk < 37 9 / 187 7 / 188 1.31 (0.48 – 3.59)

Kurkinen et al 2000 101 12.4 Low risk < 34 1 / 51 1 / 50

McDonald et al 1997* 480 24.1 All

Low risk

High risk

< 37

< 37

< 37

11 / 242

10 / 225

1 / 17

15 / 238

2 / 221

6 / 17

0.71 (0.32 – 1.57)

1.10 (0.44 – 2.75)

0.11 (0.01 – 1.09)

McGregor et al 25 30.7 All < 37 9 / 60 5 / 69 2.26 (0.71 – 7.16)

Morales et al 1994* 80 16.5 High risk

High risk

< 37

< 34

8 / 44

2 /44

16 / 36

4 / 36

0.28 (0.10 – 0.76)

0.38 (0.38 – 2.21)

Antibiotic treatment of BV in pregnancy: a meta – analysisLeitich et al. Am J Obstet Gynecol 2003;188:752 - 758

Treatment initiated early made significant difference, treatments initiated >20wks made no difference.

Systematic review of antibiotic for the treatment of bacterial vaginosis in pregnancy comparing preterm delivery rates (< 37 weeks) with any antibiotic versus placebo or no treatment.

Adapted from McDonald et al 2005

Cochrane review of all trials showing lack of evidence to support treatment

Effect of antibiotics therapy vs placebo for intermediate flora or bacterial vaginosis on the risk of preterm birth:

Adapted from McDonald et al Clindamycin Early treatment Route?

Early Rx using oral clindamycin - significant reduction

Effect of antibiotics therapy vs placebo for intermediate flora or bacterial vaginosis on the risk of preterm birth:

Ugwumadu et al 2006 (unpublished)Unpublished meta-analysis of all RCTs using early oral clindamycin showing benefit 

ConclusionsThe gestation at diagnosis and Treatment influences the outcome

Symptomatic pregnant women should be treated

Women with history of PTB or late miscarriage benefit from Rx

Treatment when indicated should be initiated early

Clindamycin appear to be more efficacious than metronidazole

Both oral & topical clindamycin probably OK if Rx given early

More trials required to clarify screening the general population

Electronic Fetal Monitoring

Is there evidence to do EFM in high risk labour?

Is there evidence to do EFM in low risk labour?

Does CTG reduce intrapartum mortality?Does CTG reduce intrapartum mortality?

Retrospective studies Retrospective studies Historical Controls – 1970sHistorical Controls – 1970s

EFM decreased SB rate by 2-3/1000 & EFM decreased SB rate by 2-3/1000 & NN death rate by 6-7/1000 (Several NN death rate by 6-7/1000 (Several studies)studies)

1) Lee & Baggish – Obstet & Gynecol 1976; 2) Shenkar 1) Lee & Baggish – Obstet & Gynecol 1976; 2) Shenkar et.al. Obstet & Gynecol 1975; 3) Edington et.al. BMJ et.al. Obstet & Gynecol 1975; 3) Edington et.al. BMJ 1975; 4) Weinraub et.al. Isr J Med Sci 1978;5) Koh et.al. 1975; 4) Weinraub et.al. Isr J Med Sci 1978;5) Koh et.al. Can Med Assoc J 1975; 6) Johnstone et.al. Lancet 1978; Can Med Assoc J 1975; 6) Johnstone et.al. Lancet 1978; 7) Hochuli Schweiz Med Wochenschr 1976;8) Lehmann 7) Hochuli Schweiz Med Wochenschr 1976;8) Lehmann et al. Geburtshilfe Frauenheilkd 1976et al. Geburtshilfe Frauenheilkd 1976

Intrapartum Fetal Heart Rate monitoring Intrapartum Fetal Heart Rate monitoring Vs Intermittent AuscultationVs Intermittent Auscultation

N =18,561 – 9 published studiesN =18,561 – 9 published studies

Higher CS rate - OR – 1.53 (1.17-2.01)Higher CS rate - OR – 1.53 (1.17-2.01)

Higher CS rate for Higher CS rate for ‘‘FDFD’’ - OR 2.55 (1.81-3.53) - OR 2.55 (1.81-3.53)

Increased IVD - OR – 1.23 (1.02-1.49)Increased IVD - OR – 1.23 (1.02-1.49)

Increased IVD for Increased IVD for ‘‘FDFD’’ – 2.50 (1.97-3.18) – 2.50 (1.97-3.18)

Decreased PNM due to fetal hypoxia – OR 0.41 Decreased PNM due to fetal hypoxia – OR 0.41 (0.17-0.98) (0.17-0.98)

Vintzelios et.al. Obstet Gynecol Vintzelios et.al. Obstet Gynecol 19951995 Haverkamp et.al. Am JO&G 1976 & 1979; Renou et.al. Am J O&G 1976; Kelso et.al. Am J Haverkamp et.al. Am JO&G 1976 & 1979; Renou et.al. Am J O&G 1976; Kelso et.al. Am J

Obstet Gynecol 1978; Wood et.al. Am J Obstet Gynecol 1981; Mac Donald et.al. Am J Obstet Gynecol 1978; Wood et.al. Am J Obstet Gynecol 1981; Mac Donald et.al. Am J O&G 1985; Neldam et.al. Eur J Obstet Gynecol1986; Luthy et.al. Obstet Gynecol. 1987; O&G 1985; Neldam et.al. Eur J Obstet Gynecol1986; Luthy et.al. Obstet Gynecol. 1987; Vintzileos et.al. Obstet Gynecol. 1993 Vintzileos et.al. Obstet Gynecol. 1993

Comparing continuous electronic fetal monitoring in labour (cardiotocography, CTG) with intermittent listening (intermittent auscultation, IA)

Alfirevic Z, Devane D, Gyte GML; Published Online: November 8, 2013Review included 13 trials involving over 37,000 women that compared continuous CTG monitoring with intermittent auscultation (listening). Most studies were not of high quality and the review is dominated by one large, well-conducted trial of almost 13,000 women who received one-to-one care throughout labour. In this trial, the membranes were ruptured artificially (amniotomy) as early as possible and oxytocin stimulation of contractions was used in about a quarter of the women.

Overall, there was no difference in the number of babies who died during or shortly after labour (about one in 300). Fits (neonatal seizures) in babies were rare (about one in 500 births), but they occurred significantly less often when continuous CTG was used to monitor the fetal heart rate.

There was no difference in the incidence of cerebral palsy, however, other possible long-term effects have not been fully assessed and need further study.

Continuous monitoring was associated with a significant increase in CS & IVD. Both procedures are known to carry the risks for mothers although the specific adverse outcomes were not assessed in the included studies.-

NEONATAL SEIZURES

PERINATAL MORTALITY

Incidence of IP still births & NN deathsIncidence of IP still births & NN deaths

To study impact of EFM on reduction of IP related To study impact of EFM on reduction of IP related deaths large numbers need to be studied 80 to deaths large numbers need to be studied 80 to 100,000.100,000.A review of IP fetal deaths 1982-2002. Mattatall et.al. 2005; A review of IP fetal deaths 1982-2002. Mattatall et.al. 2005; IJOG; IP deaths - 82/121,659 births -0.67/1000; 82 = 51 IJOG; IP deaths - 82/121,659 births -0.67/1000; 82 = 51 previable + 20 major anomaly; previable + 20 major anomaly; Viable = 11 (0.09/1000 or 0.9/10,000Viable = 11 (0.09/1000 or 0.9/10,000) ) Conclusion; EFM & rapid operative delivery Conclusion; EFM & rapid operative delivery may reduce already low rate of IP deathsmay reduce already low rate of IP deaths

Incidence of HIE in the UK 2/ 1000 – Grades 2&3 =1/1000; Incidence of HIE in the UK 2/ 1000 – Grades 2&3 =1/1000; 50% of grades II & III Die or develop CP50% of grades II & III Die or develop CP

IP – Hypoxia > Asphyxia > IP still births > HIE > NN deathsIP – Hypoxia > Asphyxia > IP still births > HIE > NN deaths

Obstetric intervention and benefit in conditions of very low prevalence

This figure shows the number of cases required in each arm of a study to detect changes in interventions and outcomes, using a 0.8 power and a type 1 error of 0.05.

To detect a doubling or halving of the CS rate from 10% requires 220 and 475 cases in each arm, respectively.

A similar doubling or halving in the incidence of low Apgar scores from around 0.9% would require 2799 and 5634 cases, and for encephalopathy 85,012 and 42,488 cases.

Mongelli suggests that the flaws inmuch of the evidence have given specialprominence to the negative aspects ofEFM, whereas the possible benefitshave been obscured by studies with insufficient

numbers.

Mongelli et al, BJOG 1997

The intervention-benefit ratio and research models

The positive predictive value (i.e. the proportion of those tested positive with the predicted outcome) can be derived from test sensitivity, specificity and disease prevalence.

If intervention occurs whenever a test is positive, the intervention rate is equivalent to the test positive rate. However, such an intervention would only be beneficial if the test result is a true positive.

The intervention-benefit ratio (IBR) can therefore be defined as the inverse of the positive predictive value.

In the context of EFM the IBR represents the number of interventions that would be necessary to prevent a case of adverse neonatal outcome:

Mongelli et al, BJOG 1997

What % of women were electronically monitored in the EFM group?What % CTG were not interpretable in the first stage? In the second stage?

Did it increase CS or IVD rate?Did the women in the IA group have FBS in labour?What was the commonest indication for FBS? i)I) Abn CTG of FHR?ii) Ii) Meconium?iii) Labour> 8 hrs?

FETAL SCALP BLOOD SAMPLING

EFM is equivalent to EFM if FBS is performed for women in labour > 8 hrs ???

OPERATIVE DELIVERIES

Entry Criteria – at ARM must have clear amniotic fluid

What was the mean cervical dilatation at ARM?Did it increase CS or IVD?

AT is of little value if ARM at < 1.5 cm shows clear fluid

How many use a partogram?

Is there evidence that partogram is useful to reduce CS? or infection? or operative delivery?

Effect of partogram use on outcomes for women in spontaneous labour at term

Lavender T, Hart A, Smyth RMD; Published Online: July 10, 2013

It has been unclear whether a partogram should be used and, if so, which design of partogram is better for women and babies.

Review authors identified six randomised controlled trials involving 7706 women in spontaneous labour at term. Two studies, with 1590 women, assessed introducing the use of a partogram versus routine care without a partogram. Four studies, involving 6116 women, compared different types of partograms.

Overall, there was no evidence from this review that using a partogram reduced or increased CS rates or had any effect on other aspects of care in labour. Where different types of partogram were compared, no design appeared better than others. A single centre study, conducted in India, however, comparing a partogram with a latent phase (composite) and one without, demonstrated more favourable outcomes for the mother and baby when the modified chart was used. It is possible that partograms may be useful in settings with poorer access to healthcare resources, as studies in Mexico and Africa also showed some reduction in caesarean section rates with partogram use and early intervention for delayed progress in labour.-

World Health Organization partograph in management of labour – WHO maternal health

and safe motherhood programme*

SummaryAs part of the Safe Motherhood Initiative, launched in 1987, theWorld Health Organization have produced and promoted a partograph with a view to improving labour management and reducing maternal and fetal morbidity and mortality. This partograph has been tested in a multicentre trial in south east Asia involving 35 484 women.

Introduction of the partograph with an agreed labour management protocol reduced both prolonged labour (from 6.4% to 3.4% of labours) and the proportion of labours requiring augmentation (from 20.7% to 9.1%)

Lancet 1994; 343: 1399-404

Four pairs of hospitals in south east Asia were invited to participate (2 pairs in Indonesia, 1 each in Thailand and Malaysia). All the centres functioned as district general hospitals in urban environments with adequate medical and midwifery staffing and suitable facilities for operative obstetric care. All were already practising active labour management including oxytocin augmentation.

The study ran for 15 months from 1 January, 1990.

During the first Five months, all centres collected data on their deliveries on a standardised form for entry onto the database held on computer at WHO headquarters in Geneva. After Five months, the WHO partograph was randomly introduced in one of each hospital pair.Ten months into the study, the partograph was introduced into the remaining hospitals and thus used in all 8 for the last 5months. No discussions were held concerning labour management or

Emergency CS fell from 9.9% to 8.3%, and intrapartum stillbirths from 0.5% to 0.3%.

Among singleton pregnancies with no complicating factors, the improved outcome was even more marked, with CS falling from 6.2% to 4.5%.The improvements took place among both nulliparous and multiparous women.

The World Health Organisation partograph clearly differentiates normal from abnormal progress in labour and identifies those women likely to require intervention. Its use in all labour wards is recommended.

Lancet 1994; 343: 1399-404

Challenges to Evidence Based Medicine There is a need for more and better research

Inadequacy of numbers

Inclusion & Exclusion criteria need to be well defined

Details of the study need to be explored (sub group meta

analysis. E.g fruits - mix of apples and pears Vs bananas cf- BV in early Vs late gestation)

Conclusions drawn must spell out the points to make note if one is to follow their conclusions (e.g. ARM at Cx <2 cm with AT)

Exclusion of studies based on reviewer’s bias? (Use of WHO partogram)

Should reviewers of Cochrane are the same as those conducted the major studies

THANK YOU