Accident Analysis and Prevention 38 (2006) 636–643

Misconceptions regarding case-control studies of bicyclehelmets and head injury

Peter Cummings a,∗, Frederick P. Rivara a, Diane C. Thompson a,1, Robert S. Thompson b

a Harborview Injury Prevention and Research Center, University of Washington, Seattle, WA, USAb Group Health Center for Health Studies, Department of Preventive Care, Group Health Cooperative of Puget Sound, Seattle, WA, USA

Received 24 June 2005; received in revised form 8 December 2005; accepted 12 December 2005

Abstract

A number of published case-control studies have reported that bicycle helmets are associated with a reduced risk of head injury and brain injuryamong bicyclists who crashed. A paper in this journal offered several criticisms of these studies and of a systematic review of these studies. Manyof those criticisms stem from misconceptions about the studies that have been done and about case-control studies in general. In this manuscriptwe review case-control study design, particularly as it applies to bicycle helmet studies, and review some aspects of the analysis of case-controldata.©

K

1

paahi

ac1dwnbtmitc

T

0d

2005 Elsevier Ltd. All rights reserved.

eywords: Bicycle helmets; Case-control studies

. Introduction

Bicycles are an important means of transportation in manyarts of the world, as well as an important source of recreationnd physical activity for many people. The majority of seriousnd fatal injuries related to bicycling involve the head. Bicycleelmets have been promoted as one means of preventing thesenjuries.

An article in this journal (Curnow, 2005) critiqued a system-tic review (Thompson et al., 1999) done by three of us regardingase-control studies (Maimaris et al., 1994; McDermott et al.,993; Thompson et al., 1989, 1996a,b) that have reported evi-ence that wearing a helmet in a bicycle crash was associatedith a reduced risk of head and facial injuries. Curnow raised aumber of points that are important to examine here. These cane categorized into questions about the importance of injurieso the head but not to the brain, the mechanism by which hel-

ets may prevent brain injury in a bicycle crash, potential biasesn the case-control studies of helmet use and head injury, andhe analysis of case-control data. (Other issues regarding bicy-le helmets and the systematic review have been previously

addressed (Thompson et al., 2004) and will not be consideredin this manuscript.)

2. Importance of head injuries

Head injury is a term used to describe injuries to the scalp,skull, and brain, while brain injury more specifically refers toinjuries that cause some degree of brain dysfunction, includingconcussion, intracranial hemorrhage, and diffuse axonal injury.While injuries to the head that result in death nearly alwaysinvolve brain injury, injuries to the scalp and skull can cause sig-nificant disability and should be prevented if possible. Injuries tothe scalp can include large lacerations, which in children some-times require general anesthesia to repair adequately. Fracturesto the vault of the skull usually heal without long-term conse-quences, however skull fractures with depression more than thewidth of the cortex may cause seizures and usually require opera-tive repair with plates (Smith and Grady, 2005). Fractures to thebase of the skull (basilar skull fractures) often cause intracra-nial bleeding and can injure the eighth cranial nerve or theossicles resulting in hearing impairment. They can also cause

∗ Corresponding author at: 250 Grandview Drive, Bishop, CA 93514, USA.el.: +1 760 873 3058.

E-mail address: peterc@u.washington.edu (P. Cummings).1

cerebrospinal fluid leaks with consequent risk of meningitis andventriculitis.

The Cochrane review regarding bicycle helmets alsoaddressed the issue of facial injury (Thompson et al., 1999),a

Retired.

001-4575/$ – see front matter © 2005 Elsevier Ltd. All rights reserved.oi:10.1016/j.aap.2005.12.007

nd reported that helmets appeared to decrease the risk of injury

P. Cummings et al. / Accident Analysis and Prevention 38 (2006) 636–643 637

to the upper and middle face (Thompson et al., 1996a). Facialinjures are common in cyclists and can require operative repair,involve cranial nerve injury, and result in cerebrospinal fluidleaks.

3. Theoretical basis for protection by bicycle helmets

Helmets for bicyclists did not evolve from helmets for sol-diers. For the latter, the mechanism of injury is penetratingtrauma from bullets, not energy transfer from blunt impact.Injuries to the brain in blunt trauma, such as those incurred in abicycle crash, occur from energy transfer to the scalp, skull, andunderlying brain. Bicycle helmets were designed with a liner toabsorb the energy transfer, not to prevent penetrating injury.

Linear as well as angular acceleration are important in headinjuries from motor vehicle crashes (Ryan et al., 1994). Energyabsorbing padding is a key element in reducing risk of braininjury (Nirula et al., 2003). Studies indicate that, unlike the mil-itary situation, penetrating injury to the head was rare amongbicyclists (Cameron et al., 1994; Ching et al., 1997). Australiadeleted the penetration test from bicycle helmet standards in1990 (Ching et al., 1997).

4. Study question

Systematic reviews, such as those of the Cochrane Collabora-tTwdhaifitTrodac

5

tcti(Hpct(ap

Haddon and colleagues in the early 1960’s (Haddon et al., 1961;McCarroll and Haddon, 1962). Since then many injury-relatedcase-control studies have been published. Textbooks (Breslowand Day, 1980; Cummings et al., 2001; Kelsey et al., 1996;Koepsell and Weiss, 2003; MacMahon and Trichopoulos, 1996;Rothman and Greenland, 1998; Rothman, 2002; Schlesselman,1982) and articles (Armenian, 1994; Roberts, 1995) havedescribed the design of case-control studies. We will brieflyreview the method here.

Imagine that we wished to know if wearing a bicycle hel-met was associated with the risk of head injury in a bicyclecrash. We might first consider a randomized controlled trial, asrandom allocation of bicyclists to helmet wearing or not is anexcellent way of forming two comparison groups that are similarwith regard to other factors that might influence the risk of headinjury in a crash. We would probably have to reject this designfor several reasons: (1) bicycle-related head injuries are uncom-mon, so a randomized trial would be large and expensive; (2)getting bicyclists to consent to random allocation of helmet usemay be difficult; (3) since there is evidence (Thompson et al.,1999) that helmets protect against head injuries, a human sub-jects committee would likely not agree that random allocationof helmet wearing is ethical.

If we cannot obtain approval or funding for a randomized trial,we might conduct a cohort study of helmet wearing and headinjury. We could enroll 4 million members of cycling clubs andaawocFt0Iiw

wttcr(4b

TRb

C

H

HNT

ion, the U.S. Preventive Services Task Force, and the Canadianask Force on Preventive Health Care, are studies and must beginith a testable question. The Cochrane review of bicycle helmetsid have such a question (Thompson et al., 1999): do “bicycleelmets reduce head, brain and facial injury for bicyclists of allges involved in a bicycle crash or fall?” In order to avoid biasn the selection of studies to be included, systematic reviewsrst start with clear inclusion and exclusion criteria, designed

o include studies relevant to the question (Egger et al., 2001).here were clear criteria for inclusion of studies in the helmet

eview, including prospective identification of cases, validationf all injuries by means of medical record review, equivalentetermination of exposure (helmet use) for cases and controls,ppropriate selection of the control group, and some attempt toontrol for possible confounding factors.

. Case-control design

Case-control studies of injury outcomes date back at leasto 1938 when Holcomb (Holcomb, 1938) published a study thatompared the prevalence of alcohol in the blood of drivers hospi-alized after a traffic crash (47%) with the prevalence of alcoholn the breath of drivers sampled on roads in Evanston, Illinois12%). The term case-control study had not yet been coined, butolcomb appreciated that if alcohol were a cause of crashes, therevalence of alcohol use would be greater among drivers whorashed compared with similar drivers who did not crash. Statis-ical methods to estimate crude (Cornfield, 1951) and adjustedMantel and Haenszel, 1959) risk ratios from case-control datappeared in the 1950’s. Formal case-control studies of driver oredestrian alcohol use and traffic crash death were published by

sk them for information about a single 100-mile ride duringyear of follow-up. Let us assume that 50% of the cyclistsear a helmet, 1% (40,000) crashed during the ride, the riskf a crash was unrelated to helmet use, and among those whorashed 1% sustained a head injury if they were not helmeted.or this hypothetical example, we assume that helmets reduced

he risk of head injury by 50% among those who crashed, i.e..5% of helmeted riders who crashed sustained a head injury.f we collected information about crashes, helmet use, and headnjuries from the cohort of 4 million cyclists, on average the dataould look like Table 1.Since we wish to estimate the association between helmet

earing and head injury in a crash, we would only need data fromhe 40,000 cyclists who crashed. In that group we could estimatehat the risk of head injury was less among helmeted cyclists whorashed compared with unhelmeted cyclists who crashed: riskatio = [A/(A + B)]/[C/(C + D)] = (100/20,000)/(200/20,000) = 0.595% confidence interval [CI] 0.39–0.64). Obtaining data frommillion cyclists or from 40,000 cyclists who crashed, would

e difficult, expensive, and time-consuming.

able 1esults from a hypothetical cohort study of helmet use and head injury amongicyclists who crashed

rashed

elmet use Did not crash Head injury No head injury Total

elmeted 1980000 [A] 100 [B] 19900 2000000ot helmeted 1980000 [C] 200 [D] 19800 2000000otal 3960000 300 39700 4000000

638 P. Cummings et al. / Accident Analysis and Prevention 38 (2006) 636–643

Table 2Results from a hypothetical case-control study of helmet use and head injuryamong bicyclists who crashed

Head injury

Helmet use Yes No Total

Helmeted [A] 100 [B] 199 299Not helmeted [C] 200 [D] 198 398

Total 300 397 697

When a study outcome is rare, a case-control design is ofteneasier to conduct, cheaper, and can obtain results faster, com-pared with a cohort study of the same question. If we could obtaininformation about all the 300 cyclists who had head injuries (thecases in cells A and C of Table 1) and just 1% of the 39,700cyclists who suffered no head injury (the controls in cells B andD), on average we would have data like those in Table 2.

From these case-control data we can estimate that amongriders who crashed, the odds ratio for a head injury wasless for a helmeted rider compared with an unhelmeted rider:odds ratio = (A/B)/(C/D) = (100/200)/(199/198) = 0.497 (95% CI0.36–0.69). The estimated odds ratio closely approximates therisk ratio from the cohort data. The confidence intervals areonly modestly wider in the case-control study, which used infor-mation about only 697 riders, compared with the cohort study,which required information from at least the 40,000 riders whocrashed. The case-control design can estimate risk ratios moreefficiently than a cohort study of the same association when theoutcome is rare.

6. Control selection in case-control studies

In the hypothetical data in Table 2, the controls were a randomsample of the non-cases. In a case-control study, the controls areused to estimate the prevalence of the study exposure (helmetuMoelTdf

ewsawittwa

b

In one case-control study of bicycle helmets (Thompson etal., 1989), the authors gathered information from two controlgroups. Among non-case (no head injury) bicyclists who cameto an emergency department the prevalence of helmet use was23.8%. The investigators used a second control group that wasrandomly sampled from among members of a large health main-tenance organization, frequency matched to the cases on ageand zip code; the prevalence of helmet wearing in this group ofcyclists who crashed was 23.3%. Thus both control groups hada similar prevalence of helmet use in a crash and both preva-lences were greater than the helmet-wearing prevalence of thehead-injured cases, 7.2%.

7. Choice of outcome in studies of helmet effectiveness

Instead of estimating the association between helmet wearingand the risk of head injury in a crash, we might want to study theassociation between helmet wearing and the risk of brain injuryin a crash. If brain injury was the outcome of interest, then thecases would be cyclists who crashed and suffered a brain injury.Who should be the controls? To help answer this question, let’sreturn to the cohort data of Table 1, but this time assume that20% of the 300 people with a head injury had a brain injury(i.e. 60 cyclists) and that helmets reduced the risk of both braininjury and other head injuries by 50%. The revised data for thec

fw[0stTsgiopaaorbr[

TRb

H

H

HNT

se in this case) in the population from which the cases arose.any case-control studies are not able to select a random sample

f non-cases; for example, a registry of cycling crashes does notxist. Instead, investigators try to pick a control group that isikely to represent the exposure prevalence of the non-cases.he textbooks about case-control studies that we cited earlieriscuss control selection strategies and further details can beound in a series of articles (Wacholder et al., 1992, 1992a,b).

Most case-control studies of bicycle helmets and head injurystimated the prevalence of helmet wearing among all cyclistsho crashed by interviewing cyclists who crashed and who sub-

equently came to an emergency department or were admitted tohospital for injuries that did not involve the head. Thus cyclistsho suffered a fractured wrist or a lacerated knee, but no head

njury, were used to represent the non-cases. So long as presen-ation to a hospital by non-cases is unrelated to helmet wearing,his group of injured cyclists should fairly represent the helmet-earing prevalence of all cyclists who crashed (Cummings et

l., 1998, 2001).There is some evidence that this choice of controls may

e reasonable in the case-control studies of bicycle helmets.

yclists who crashed would look like those in Table 3.From these cohort data we can estimate that the risk ratio

or brain injury among helmeted riders who crashed, comparedith unhelmeted riders who crashed, was [A1/(A1 + A2 + B)]/

C1/(C1 + C2 + D)] = (20/20,000)/(40/20,000) = 0.5 (95% CI.29–0.85). Now let us try to use Table 3 data for a case-controltudy. The brain-injured people are the cases and we assumehat we can locate all of them at a hospital or the morgue.he people in the no-head-injury group are all eligible forelection as controls; let us assume we sample 1% of thatroup for a total of 397 cyclists. What about the 240 peoplen the other-head-injury column? They are non-cases for thisutcome. Since our goal is to estimate the helmet-wearingrevalence in all of the non-cases (those with no head injurynd those with head injury but no brain injury), we shouldlso pick a 1% sample of the other-head-injury cyclists; aboutne cyclist from cell A2 and two from cell C2. Doing this willesult in the data in Table 4. The approximate risk ratio forrain injury among helmeted riders compared with unhelmetediders can be estimated from the odds ratio [A1/(A2 + B)]/C1/(C2 + D)] = (20/200)/(40/200) = 0.5 (95% CI 0.27–0.91).

able 3esults from a hypothetical cohort study of helmet use and brain injury amongicyclists who crashed

ead injury

elmet use Brain injury Other head injury No head injury Total

elmeted [A1] 20 [A2] 80 [B] 19900 20000ot helmeted [C1] 40 [C2] 160 [D] 19800 20000otal 60 240 39700 40000

P. Cummings et al. / Accident Analysis and Prevention 38 (2006) 636–643 639

Table 4Results from a hypothetical case-control study of helmet use and brain injuryamong bicyclists who crashed

Head injury

Helmet use Brain injury Other head injury No head injury Total

Helmeted [A1] 20 [A2] 1 [B] 199 220Not helmeted [C1] 40 [C2] 2 [D] 198 240

Total 60 3 397 460

But what if we sampled the data in Table 3 usingcyclists who crashed and came to emergency departments?Assume that we found all the people with brain injuriesin emergency departments (60 cyclists) and all the peoplewith other head injuries (240 cyclists). Most other cyclistswho crashed would have no important injury and so theywould not come to an emergency department; assume only1% of those with injuries not to the brain or head (397cyclists) would come to an emergency department. Given theseassumptions, the data would look, on average, like those inTable 5. Now if we included all 240 of the other-head-injurypatients in the control group, the estimated risk ratio becomes[A1/(A2 + B)]/[C1/(C2 + D)] = (20/299)/(40/398) = 0.64 (95% CI0.35–1.15). This estimate is wrong. Our mistake was to ignore aprinciple of control selection: the probability of selecting eachnon-case should not be related to the exposure (helmet wear-ing in this instance) of interest. In our hypothetical data, helmetwearing reduced the risk of any head injury; thus the prevalenceof helmet wearing was less among the other-head-injury controls(33% of bicyclists in the other head injury column in Table 5)than among all controls (50% among all the persons withoutbrain injury in Table 3). The emergency department bicyclistswithout brain injury included 100% of bicyclists with other headinjuries, but only 1% of bicyclists without head injuries. Includ-ing all the other-head-injury bicyclists in Table 5 in the controlgroup is a form of selection bias. It would be correct to includetshs

lb0

TRai

H

H

HN

T

comparison will result in only trivial bias in an actual bicy-cle helmet case-control study, because this group will representonly a small proportion of all non-case cyclists (Cummingset al., 1998). We can also estimate the risk ratio for helmetwearing and other head injuries by ignoring the few subjectswho had a brain injury. We can estimate both risk ratios simul-taneously using multinomial (polytomous) logistic regression:both are 0.50 using this method (Hosmer and Lemeshow, 2000;Greenland and Finkle, 1996). The potential benefits and short-comings of using emergency department patients as controlshave been reviewed in more detail elsewhere (Cummings et al.,1998, 2001).

8. Misconceptions about case-control studies

Having reviewed case-control design, we now discuss someof the misconceptions related to case-control studies in the paperby Curnow (Curnow, 2005).

8.1. Misconception 1: studies should be rejected if theymay be biased

Curnow suggested that the studies included in the Cochranereview “are vulnerable to bias” (Curnow, 2005). All studies, evenrandomized controlled trials, are vulnerable to bias. In evaluat-ing any study, we need to consider what may be the size anddaadccoiatacs

8

bWsdmagwdFtiae

hese other head-injured persons in the control group if we couldelect them with the same probability as the controls without anyead injury. But we would not know this probability in an actualtudy of bicyclists seen in emergency departments.

The appropriate risk ratio can be approximated with theeast bias from Table 5 data by ignoring the other-head-injuryicyclists: (A1/B)/(C1/D) = (20/199)/(40/198) = 0.50 (95% CI.27–0.91). Omitting the other-head-injury bicyclists from the

able 5esults from a hypothetical case-control study of helmet use and brain injurymong bicyclists who crashed, based on information from injured bicyclists seenn emergency departments

ead injury

elmet use Brain injury Other head injury No head injury Total

elmeted [A1] 20 [A2] 80 [B] 199 299ot helmeted [C1] 40 [C2] 160 [D] 198 398

otal 60 240 397 697

irection of any bias. One of the reasons to conduct a system-tic review or meta-analysis is to examine how the estimates ofssociation vary between studies that will usually differ in manyetails of their execution. As we noted earlier, cohort and case-ontrol studies are more vulnerable to bias due to confoundingompared with randomized controlled trials, but we cannot relyn randomized trials to answer all questions. Case-control stud-es have made important contributions to public health on topicss diverse as the association between alcohol use and death in araffic crash (Borkenstein et al., 1964), the relationship betweenspirin and Reyes syndrome (Hurwitz et al., 1987), and the asso-iation between prone sleep position and sudden infant deathyndrome (Guntheroth and Spiers, 1992).

.2. Misconception 2: controls must be a random sample

Curnow suggested that the only valid control group woulde a random sample of all potential controls (Curnow, 2005).hile random sampling of controls is one method of avoiding

election bias, it is not the only method. One bike helmet studyid use randomly sampled controls (conditional on frequencyatching to the cases) from a health maintenance organization

nd found a prevalence of helmet use similar to that of emer-ency department controls (Thompson et al., 1989). Comparedith a random sample of all bicyclists who crashed, emergencyepartment controls may actually reduce bias from two sources.irst, confounding by crash severity may be reduced because

he controls, like the cases, were hurt sufficiently to seek med-cal care. Second, bias in recall of helmet use may be reduced,s both cases and controls have a prominent recent event (themergency visit) to help their memory regarding helmet use and

640 P. Cummings et al. / Accident Analysis and Prevention 38 (2006) 636–643

both were interviewed within a similar time interval after thecrash.

8.3. Misconception 3: confusion regarding the studyquestion

Curnow expressed concern that helmet-wearing cyclists maybe more careful or less careful than other cyclists (Curnow,2005). This may be true, but it has little relevance to the bicyclehelmet studies that were summarized in the systematic review(Thompson et al., 1999). Those studies sought to assess whetherhelmet wearing was associated with head injury among cyclistswho crashed. All cyclists crashed in the five studies summarizedand the use of injured controls helped ensure that most crasheswere not trivial. It is possible that crash severity differed by hel-met use, but one study reported that adjusting for hitting a motorvehicle, estimated bicycle speed, type of surface hit, and damageto the bicycle made little difference to the odds ratio estimate(Thompson et al., 1996b); this implies either that the case andcontrol groups differed little in regard to these factors, or thesefactors were not associated with helmet use, or both.

8.4. Misconception 4: misunderstanding the study outcome

Curnow stated that the bicycle helmet studies assumed thatcyclists could only injure their brain by hitting their heads(ir

8c

actofrwhmio

hhfadotoroh

few case-control studies will have a control group that satisfiesall three criteria perfectly. For this reason, investigators whoconduct case-control studies often devote considerable effort toprevent or control for bias due to confounding.

Confounding arises when the true association between anexposure (helmet use) and an outcome (brain injury) is distortedby the presence of some other factor which is distributed differ-ently among the exposed and unexposed, and is also related tothe occurrence of the outcome. Textbooks (Breslow and Day,1980; Cummings et al., 2001; Kelsey et al., 1996; Koepselland Weiss, 2003; MacMahon and Trichopoulos, 1996; Rothmanand Greenland, 1998; Rothman, 2002; Schlesselman, 1982)devote considerable attention to confounding and additionalinformation can be found in several articles (Greenland andMorgenstern, 2001; Maldonado and Greenland, 1993; Mickeyand Greenland, 1989). For example, in studies of bicycle hel-mets and brain injuries, imagine that helmeted riders crashedat a slower speed than unhelmeted riders and that crashing at aslower speed reduced the risk of brain injury; if this were so,then crash speed could confound (distort or bias) the estimatedrisk ratio for the effects of helmet wearing on the outcome ofbrain injury. In this example, failure to account for confoundingby crash speed would result in a risk ratio that would exaggerateany protective effect of helmets.

In case-control studies there are three basic strategies used tocontrol for possible confounding by crash speed. One methodiswct(a

seteabBccwaabtvimwBccts

Curnow, 2005). To the contrary, all the studies selected head-njured persons (including those with brain injury) as cases,egardless of how the injury occurred in the crash.

.5. Misconception 5: failure to understand confounding inase-control studies

Curnow criticized (Curnow, 2005) one study (Thompson etl., 1996b) because the authors failed to show that the cases andontrols had equal probabilities of hitting their heads. We suspecthat no study could show this. It would be hard to measure theutcome of striking the head in any study. For cyclists with skullractures, scalp lacerations, or contusions of the scalp, we couldeasonably infer that they struck their heads. But some cyclistsho hit their heads with little force may be unaware that thisappened. If helmets prevent head injury, some helmeted cyclistsay not know if they struck their heads. Some patients with brain

njuries may have no memory of the event and, as Curnow pointsut, brain injury can occur without a blow to the head.

We suspect that Curnow’s concern is that in an ideal study theelmeted and unhelmeted riders (not cases and controls) wouldave equal probability of having a head or brain injury, asiderom any effect of helmet use. To put this a little differently, incase-control study the ideal controls would have: (1) the sameistribution of the main study exposure (in this instance the usef helmets) as the population from which the cases arose, (2)he same distribution of factors that influenced the likelihoodf the exposure (helmet use), and (3) also be like the cases inegard to other factors that would influence the risk of the studyutcome (aside from any causal relationship those factors mightave with the exposure) (Koepsell and Weiss, 2003). In practice,

s restriction; cases and controls could all be restricted to high-peed (or low-speed) crashes only. If the restricted speed rangeere sufficiently narrow, this would eliminate crash speed as a

onfounder. The second method is matching: one or more con-rols could be selected to match each case in regard to crash speedusing a sufficiently narrow range) and this matching would beccounted for in the statistical analysis.

The third method is statistical adjustment, usually in regres-ion. Statistical adjustment of the risk ratio of interest is anxcellent way of examining the data to see if there are impor-ant differences between the cases and controls that bias thestimated association. Authors sometimes assess whether a vari-ble is a confounder by examining p-values for the associationetween the potential confounding variable and the outcome.ut a large (i.e. not significant) p-value may miss importantonfounding. The p-value might be large if the variable is veryommon or rare in the data, or if the outcome is uncommon, evenhen the variable is a confounder (Lang et al., 1998; Mickey

nd Greenland, 1989). The p-value may also be large becausevariable’s relationship with the outcome is also confounded

y other variables; the addition of other variables may unmaskhe confounding nature of a potential confounder. A small p-alue may not indicate confounding by a variable if the variables not related to the exposure. Instead of examining p-values,

any analysts examine what happens to the risk ratio estimatehen adjustment is made for a potential confounding variable.y adjusting we directly examine whether the variable actuallyonfounds the association of interest; if the estimated risk ratiohanges little with adjustment, then the variable does not distorthe association of interest. If the estimated risk ratio changes toome degree, then there is at least some confounding present

P. Cummings et al. / Accident Analysis and Prevention 38 (2006) 636–643 641

and, depending on the amount of change, we may adjust for thisvariable. This approach, with many details, is well described onpages 255–259 in a textbook (Rothman and Greenland, 1998).

In one study, three of us (Thompson et al., 1996b) used statis-tical adjustment with logistic regression to assess confounding.The published manuscript presented crude (unadjusted) riskratio estimates as well risk ratios adjusted for both age andwhether the crash was with a motor vehicle. This adjustmentrevealed little evidence of confounding by these two variables;the crude risk ratio estimate was 0.32 for head injury and theadjusted risk ratio was 0.31. The publication reported that therewas no important further change in the risk ratio when it wasadjusted for sex, education, income, riding experience, hospitalattended, estimated bicycle speed, type of surface hit, or dam-age to the bicycle. Thus the study presented evidence that thehelmeted and unhelmeted riders were either similar in regard toseveral variables or the outcomes were unrelated to these vari-ables, or both were true.

No study can be shown to be perfectly free of possible con-founding. No matter how many variables were measured andexamined, there is always the possibility of some residual con-founding because a variable was not measured and used inthe analysis, or because a confounding variable was measuredwith error, or because a measured variable was not adjusted forin a manner that removed all of its confounding influence. Ininterpreting estimates of association from any study, even a ran-did

8e

e0a(ttisth

tWuB1fwodter

Table 6Results from an actual case-control study of helmet use, brain injury, and otherhead injury, among bicyclists who crashed, based on information from injuredbicyclists seem in emergency departments

Head injury

Helmet use Brain injury Other head injury No head injury Total

Hard shell 23 79 741 843Soft shell 22 40 433 495No shell/foam 14 37 281 332No helmet 141 394 1137 1672

Total 200 550 2592 3342

than expected by chance only. A p-value of 0.05 or greater sug-gests that the observed difference could reasonably be attributedto chance alone; i.e., the difference arose solely due to the finitesample sizes of the studies. A test that the odds ratios 0.12 and0.35 for bicycle helmet effects were the same yielded a p-valueof 0.08; by this standard the difference in these odds ratios mightbe due to chance. These results do not lend support to Curnow’ssuggestion that helmet protectiveness has decreased over time.

In the 1996 study (Thompson et al., 1996b), three of usreported that the adjusted odds ratio for a brain injury was 0.17for hard shell helmets, 0.30 for soft shell helmets, and 0.36 forno shell or foam helmets. A p-value for a test that these oddsratios were from populations with the same true odds ratios wasnot statistically significant (p = 0.5); this means there was littlestatistical evidence that the three odds ratios differed from eachother for reasons other than chance. To put this differently, if thethree helmet types were equally protective, the observed differ-ences might have easily arisen by chance in the study sample.Any protective effects that helmets offer against brain injurymay vary by helmet type, but the evidence for this is currentlyweak; further studies of differences by helmet type would be auseful addition to our current knowledge.

8.7. Misconception 7: treating an injury outcome group ascontrols

ciwrih

TCo

H

HSNA

Th

omized trial, investigators and readers have to assess how likelys it that the results may be confounded, and what might be theirection and size of any confounding bias.

.6. Misconception 6: interpreting differences in risk ratiostimates without considering the role of chance

The reported odds ratio for brain injury among helmet wear-rs compared with those not wearing helmets was 0.12 (95% CI.04–0.40) in a study published in 1989 (Thompson et al., 1989)nd was 0.35 (95% CI 0.25–0.48) in a study published in 1996Thompson et al., 1996b). Curnow (Curnow, 2005) suggestedhis was evidence that helmets have become less effective overime due to less use of hard shell helmets. This interpretations possible, but it is also possible that these two odds ratios areimply two different estimates of the same true association inhe study populations; i.e., they differ by chance because studiesave finite sample sizes.

When a study is repeated, the estimate of association fromhe first study may not be exactly replicated in the second study.

e can formally test whether two or more odds ratios are similarsing statistical tests (Altman and Matthews, 1996; Altman andland, 2003; Egger et al., 2001; Matthews and Altman, 1996a,996b); the null hypothesis is that the observed odds ratios aroserom populations in which the true (but unobserved) odds ratiosere the same. The alternative hypothesis is that the observeddds ratios arose from populations in which the true odds ratiosiffered. A p-value less than 0.05 for a test of the similarity ofwo (or more) odds ratios is commonly interpreted as statisticalvidence rejecting the hypothesis of no difference in the oddsatios; i.e., the tested odds ratios differed by an amount greater

The data regarding brain injuries, other head injuries, andontrols from Thompson’s paper (Thompson et al., 1996b) aren Table 6. The other-head-injury column excludes personsith brain injuries. Using odds ratios from multinomial logistic

egression, we generated crude risk ratio estimates for both brainnjury and for other head injuries, associated with wearing eachelmet type compared with not wearing a helmet (Table 7). The

able 7rude risk ratios and 95% confidence intervals for brain injury or for head injuryther than brain injury, by helmet type, from data in Table 6

elmet type Brain injury Other head injury

ard shell 0.25 (0.160.39) 0.31 (0.24–0.40)oft shell 0.41 (0.26–0.65) 0.27 (0.19–0.38)o shell/foam 0.40 (0.23–0.71) 0.38 (0.26–0.55)ny helmet 0.33 (0.24–0.45) 0.31 (0.25–0.38)

hese are risk ratios for each outcome among bicyclists wearing each type ofelmet compared with no helmet.

642 P. Cummings et al. / Accident Analysis and Prevention 38 (2006) 636–643

Table 8Crude odds ratios and 95% confidence intervals for brain injury generated byCurnow’s method

Helmet type Brain injury

Hard shell 0.81 (0.49–1.34)Soft shell 1.54 (0.88–2.68)No shell/foam 1.06 (0.56–2.01)Any helmet 1.06 (0.74–1.51)

risk ratio estimates in Table 7 show some variation, but formaltests of any difference in the risk ratios by helmet type (p = 0.2)or by outcome type (p = 0.8) suggest that the differences foundcould easily be due to chance.

Curnow reported different ratios from the same data in Table 6(see Table 1 of Curnow’s paper (Curnow, 2005)). We showsome of these, with confidence intervals that we have added,in Table 8. Curnow interpreted these odds ratios as represent-ing the effects of helmet wearing on the outcome of brain injury(Curnow, 2005). They were produced from Table 6 data by ignor-ing the controls who had no head injury. Instead, Curnow dividedthe odds of each helmet type versus no helmet among brain-injured cyclists, by the same odds among cyclists with otherhead injuries. In essence, the cyclists with other head injurieswere treated as controls. The odds ratios in Table 8 do not esti-mate the effect of helmet use on brain injury. They estimatewhether helmets are more or less effective against brain injurythan against head injury. The odds ratio of 0.81 for hard shellhelmets estimates that a cyclist who crashed with a hard shellhelmet was less likely to sustain a brain injury, compared withanother type of head injury, than a cyclist who crashed without ahelmet. Without other information this odds ratio cannot tell usif hard shell helmets protect against brain injury, are unrelatedto the risk of brain injury, or actually increase the risk of braininjury; the odds ratio only suggests that users of helmets haveless risk of brain injury compared with their risk of other headihwi

obWiTfroswb

9

r

effectiveness of bicycle helmets in preventing head and braininjuries among riders who crash, case-control studies are ascientifically valid alternative design. The case-control stud-ies conducted to date, summarized in the Cochrane systematicreview (Thompson et al., 1999), provide evidence for a protec-tive effect of helmets in preventing injuries to the brain, head,and face among bicycle riders who crash. This review of theproper application of case-control methodology indicates thatCurnow’s conclusions (Curnow, 2005) are based on a numberof misconceptions that we have attempted to clarify.

References

Altman, D.G., Bland, J.M., 2003. Interaction revisited: the difference betweentwo estimates. BMJ 326, 219.

Altman, D.G., Matthews, J.N., 1996. Statistics notes. Interaction 1: hetero-geneity of effects. BMJ 313, 486.

Armenian, H.K. (Ed.), 1994. Applications of the case-control method. Epi-demiol. Rev. 16 (1).

Borkenstein, R.F., Crowther, R.F., Shumate, R.P., Ziel, W.B., Zylman, R.,1964. The Role of the Drinking Driver in Traffic Accidents. IndianaUniversity, Department of Police Administration, Bloomington, Indiana,pp. 1–245.

Breslow, N.E., Day, N.E., 1980. Statistical Methods in Cancer Research,vol. 1. The Analysis of Case-Control Studies. International Agency forResearch on Cancer, Lyon, France.

C

C

C

C

C

C

E

G

G

G

H

H

H

H

njuries. This design has no useful application to the bicycleelmet data, as we can easily estimate the association of helmetearing with each type of head injury outcome, as we have done

n Table 7.The type of case-control study design Curnow employed has

ccasionally been used and its merits (or lack of merit) haveeen described on pages 389–391 of a textbook (Koepsell andeiss, 2003). The additional analysis in Table 8 adds no new

nformation; the odds ratio of 0.81 for hard shell helmets inable 8 can be derived by dividing the brain injury risk ratioor hard shell helmets in Table 7 by the other head injury riskatio for hard shell helmets in Table 7: 0.25/0.31 = 0.81. Thedds ratios in Table 8, with their wide confidence intervals, areimply an inelegant way of showing that we cannot distinguishith available data whether the apparent protection offered byicycle helmets is greater for brain injuries or other head injuries.

. Conclusions

Due to the practical difficulties involved in conducting largeandomized controlled trials or cohort studies to estimate the

ameron, M.C., Finch, C., Vulcan, P., 1994. The protective performance ofbicycle helmets introduced at the same time as the bicycle helmet wearinglaw in Victoria. Australian Road Research Board Ltd., Victoria, Australia.

hing, R.P., Thompson, D.C., Thompson, R.S., Thomas, D.J., Chilcott, W.C.,Rivara, F.P., 1997. Damage to bicycle helmets involved with crashes.Accident Anal. Prev. 29, 555–562.

ornfield, J., 1951. A method of estimating comparative rates from clini-cal data. Applications to cancer of the lung, breast, and cervix. J. Natl.Cancer. Inst. 11, 1269–1275.

ummings, P., Koepsell, T.D., Roberts, I., 2001. Case-control studies ininjury research. In: Rivara, F.P., Cummings, P., Koepsell, T.D., Grossman,D.C., Maier, R.V. (Eds.), Injury Control: A Guide to Research and Pro-gram Evaluation. Cambridge University Press, New York, NY, pp. 139–156.

ummings, P., Koepsell, T.D., Weiss, N.S., 1998. Studying injuries withcase-control methods in the emergency department. Ann. Emerg. Med.31, 99–105.

urnow, W.J., 2005. The Cochrane Collaboration and bicycle helmets. Acci-dent Anal. Prev. 37, 569–573.

gger, M., Smith, G.D., Altman, D.G., 2001. Systematic reviews in healthcare: meta-analysis in context. BMJ Publishing Group, London.

reenland, S., Finkle, W.D., 1996. A case-control study of prosthetic implantsand selected chronic diseases. Ann. Epidemiol. 6, 530–540.

reenland, S., Morgenstern, H., 2001. Confounding in health research. Annu.Rev. Public Health 22, 189–212.

untheroth, W.G., Spiers, P.S., 1992. Sleeping prone and the risk of suddeninfant death syndrome. JAMA 267, 2359–2362.

addon Jr., W., Valien, P., McCarroll, J.R., Umberger, C.J., 1961. A con-trolled investigation of the characteristics of adult pedestrians fatallyinjured by motor vehicles in Manhatten. J. Chron. Dis. 14, 655–678.

olcomb, R.L., 1938. Alcohol in relation to traffic accidents. J. Am. Med.Assoc. 111, 1076–1085.

osmer, D.W., Lemeshow, S., 2000. Applied Logistic Regression, 2nd ed.John Wiley & Sons, New York, pp. 260–273.

urwitz, E.S., Barrett, M.J., Bregman, D., Gunn, W.J., Pinsky, P., Schon-berger, L.B., Drage, J.S., Kaslow, R.A., Burlington, D.B., Quinnan, G.V.,et al., 1987. Public health service study of Reye’s syndrome and medi-cations. Report of the main study. JAMA 257, 1905–1911.

P. Cummings et al. / Accident Analysis and Prevention 38 (2006) 636–643 643

Kelsey, J.L., Whittemore, A.S., Evans, A.S., Thompson, W.D., 1996. Methodsin Observational Epidemiology, 2nd ed. Oxford University Press, NewYork, pp. 188–243.

Koepsell, T.D., Weiss, N.S., 2003. Epidemiologic Methods: Studying theOccurrence of Illness. Oxford University Press, New York, pp. 105–108,247–280, 374–402.

Lang, J.M., Rothman, K.J., Cann, C.I., 1998. That confounded P-value [edi-torial]. Epidemiology 9, 7–8.

MacMahon, B., Trichopoulos, D., 1996. Epidemiology: Principles and Meth-ods, 2nd ed. Little, Brown, Boston, pp. 229–302.

Maimaris, C., Summers, C.L., Browning, C., Palmer, C.R., 1994. Injurypatterns in cyclists attending an accident and emergency department: acomparison of helmet wearers and non-wearers. BMJ 308, 1537–1540.

Maldonado, G., Greenland, S., 1993. Simulation study of confounder selec-tion strategies. Am. J. Epidemiol. 138, 923–936.

Mantel, N., Haenszel, W., 1959. Statistical aspects of the analysis of datafrom retrospective studies. J. Natl. Cancer Inst. 22, 719–748.

Matthews, J.N., Altman, D.G., 1996a. Statistics notes. Interaction 2: compareeffect sizes not P values. BMJ 313, 808.

Matthews, J.N., Altman, D.G., 1996b. Interaction 3: how to examine hetero-geneity. BMJ 313, 862.

McCarroll, J.R., Haddon Jr., W., 1962. A controlled study of fatal automobileaccidents in New York City. J. Chron. Dis. 15, 811–826.

McDermott, F.T., Lane, J.C., Brazenor, G.A., 1993. The effectiveness ofbicyclist helmets: a study of 1710 casualties. J. Trauma 34, 834–845.

Mickey, R.M., Greenland, S., 1989. The impact of confounder selection cri-teria on effect estimation. Am. J. Epidemiol. 129, 125–137.

Nirula, R., Kaufman, R., Tencer, A., 2003. Traumatic brain injury and auto-motive design: making motor vehicles safer. J. Trauma 55, 844–848.

Roberts, I., 1995. Methodologic issues in injury case-control studies. InjuryPrev. 1, 45–48.

R

Rothman, K.J., Greenland, S., 1998. Modern Epidemiology, 2nd ed.Lippincott-Raven, Philadelphia, p. 62, 93–161, 255–259.

Ryan, G.A., McLean, A.J., Vilenius, A.T., Kloeden, C.N., Simpson, D.A.,Blumbergs, P.C., Scott, G., 1994. Brain injury patterns in fatally injuredpedestrians. J. Trauma 36, 469–476.

Schlesselman, J.A., 1982. Case-Control Studies: Design, Conduct, Analysis.Oxford University Press, New York.

Smith, M.L., Grady, M.S., 2005. Neurosurgery. In: Pollock, R.E. (Ed.),Schwartz’s Principles of Surgery. McGraw-Hill, New York.

Thompson, D.C., Nunn, M.E., Thompson, R.S., Rivara, F.P., 1996a. Effective-ness of bicycle safety helmets in preventing serious facial injury. JAMA276, 1974–1975.

Thompson, D.C., Rivara, F.P., Thompson, R., 1999. Helmets for preventinghead and facial injuries in bicyclists. Cochrane Database Syst. Rev. (4)(Art. No.: CD001855, DOI: 10.1002/14651858.CD001855).

Thompson, D.C., Rivara, F.P., Thompson, R., 2004. Helmets for prevent-ing head and facial injuries in bicyclists. Available at: http://www.cochranefeedback.com/cf/cda/citation.do?id=9316#931612/7/2005.

Thompson, D.C., Rivara, F.P., Thompson, R.S., 1996b. Effectiveness of bicy-cle helmets in preventing head injuries: a case-control study. JAMA 276,1968–1973.

Thompson, R.S., Rivara, F.P., Thompson, D.C., 1989. A case-control studyof the effectiveness of bicycle safety helmets. N. Engl. J. Med. 320,1361–1367.

Wacholder, S., McLaughlin, J.K., Silverman, D.T., Mandel, J.S., 1992. Selec-tion of controls in case-control studies. Part I. Principles. Am. J. Epi-demiol. 135, 1019–1028.

Wacholder, S., Silverman, D.T., McLaughlin, J.K., Mandel, J.S., 1992a. Selec-tion of controls in case-control studies. Part II. Types of controls. Am. J.Epidemiol. 135, 1029–1041.

Wacholder, S., Silverman, D.T., McLaughlin, J.K., Mandel, J.S., 1992b. Selec-

othman, K.J., 2002. Epidemiology: An Introduction. Oxford University

Press, New York.

tion of controls in case-control studies. Part III. Design options. Am. J.Epidemiol. 135, 1042–1050.