The out-of-hospital validation of the CCR rule by paramedics

Ref: Vaillancourt C et al. The Out-of- Hospital Validation of the Canadian C-Spine Rule by Paramedics. Ann of Emerg Med Nov 2009;54(5):663-671

CRITICAL APPRAISAL Dineo Moiloa Nov 2009

Introduction

• > 1 million trauma pts w suspected C-spine injuries

• 2 % significant injuries• Problems w immobilisation, methods not

evidence-based• No improvement in pt outcome (Cochrane)• Validation study for CCR in pre-hospital setting• NOTE: Follows on landmark article (CCR vs

NEXUS in hospital)

Method of study

• Prospective cohort study: 2002-2006• 7 locations in 3 Canadian locations• 1949 enrolled patients : Neck trauma• Inclusion and exclusion criteria• Median age=39 (26-52)• 50.8% females, 49% males• 62.5 % MVA• Trained EMS vs Investigators (CCR)• Primary oucome : ID of acute cervical spine injury AND performance

characteristics of the interpretation of CCR by paramedics• Effect of CCR in clinical practise : Less Xrays

Canadian C-Spine Rule

Flow of patients recruited in study

Results 320 ‘indeterminate’cases

Investigators (95% CI) Paramedics (95% CI)

Results of assessment

Injury No Injury Injury No Injury

Positive no. 12 924 12 1204

Negative no. 0 693 0 731

Sensitivity % 100 (74-100) 100 (74-100)

Specificity % 42.9 (40-45) 37.7 (36-40)

NPV % 100 (99-100) 100 (99-100)

Results

• Performance characteristics of the interpretation of the rule by paramedics

- 5 point Likert scale : k value 0f 0.93- Very uncomfortable/uncomfortable :9.5%- Comfortable/ V comfortable : 81.7%

• Effect of rule : CCR- 62% of pts would have been immobilised (vs 100% acc to local EMS protocol)

What is the clinical question?

Can paramedics apply the Canadian C-Spine Rule in alert, stable, cooperative, blunt trauma patients to reserve spinal immobilisation for high risk patients while avoiding immobilisation for low risk patients?

Well addressed by the paper

Validation study for CCR in the pre-hospital setting

What type of study?

• Prospective cohort study• ‘Gold standard’?• Comparing CCR with local protocol- which the

researchers already regard as ‘unfounded’ and ‘not evidence-based’.

• Would results have been different if another clinical decision rule was used?

What do we know about CCR already?

Which clinical decision rule is better at recognising cervical spine injury, the Canadian C-spine rule or the NEXUS low-risk criteria rule?

Reference : Stiell IG et al. The Canadian C-spine rule vs the NEXUS Low-Risk Criteria in Patients with Trauma. N Engl J Med Dec 25, 2003; 349:250-8

Results(7438 patients)Assessment results

Injury (CCR) No Injury (CCR) Injury (Nexus) No Injury (Nexus)

Positive (no.) 161 3995 147 4599

Negative (no.) 1 3281 15 2677

Sensitivity (%) 99.4 (95% CI,96-100)

90.7 (95% CI, 85-94)

Specificity (%) 45.1 (95% CI, 44-46)

36.8 (95% CI, 36-38)

NPV (%) 100 99.4

Methodological quality of study

• Was the study original?• Who is the study about?• Was the design of the study sensible?• Was systematic bias avoided or minimised?• Was the study large enough, and continued

long enough, to make results credible?

Was the study original

• Original research• Reference : previous similar studies Up to date refs (Last 5 yrs)• Primary research- validation of clinical

decision rule in pre-hospital setting

Was the study design sensible?

• Specific intervention :CCR • Compared with? Current gold standard

already deemed inadequate• Outcome measured?- Identification of clinically-significant C-spine

injuries- Paramedics interpretation and application of

the rule- % of immobilisations potentially avoided

Flow of patients recruited in study

Bias

• Canadian researchers and most medics in Canada comfortable w CCR

• No control group• Selection of participants (at discretion of

paramedics)- Paramedics unlikely to recruit severely injured

patients into study• ED doctors widely used CCR in hospital to

‘clear’ C-spine- Would they have immoblsd more patients had they used other clinical decisions

Duration and size of study

• 2002-2006 : Long enough?• Could have continued longer to get a larger

sample size• Could the paramedics have missed more

injuries if the sample was larger? (Ref other prehospital studies)

• Lots of ‘indeterminate’ cases= 320-Misinterpretation of the rule?• Adequacy and completeness of follow up?

Flow of patients recruited in study

Results 320 ‘indeterminate’cases

Investigators (95% CI) Paramedics (95% CI)

Results of assessment

Injury No Injury Injury No Injury

Positive no. 12 924 12 1204

Negative no. 0 693 0 731

Sensitivity % 100 (74-100) 100 (74-100)

Specificity % 42.9 (40-45) 37.7 (36-40)

NPV % 100 (99-100) 100 (99-100)

Results

• 100% sensitivity and NPV- Very wide confidence intervals - Researchers attribute it to small sample size- Could the paramedics have missed more injuries if the

sample was larger• 320 indeterminate cases- Paramedics uncomfortable rotating the neck- All those revealed no significant injuries- Would there have been more false positives if they were

included in the analysis

Reliability of Paramedic’s interpretation of the rule

• 5-point Likert scale : widely used and acceptable as a tool

-Mostly comfortable/very comfortable using the rule

BUT 320 ‘indeterminate’ cases- paramedics failed to rotate the neck in cases where it was safe to do so

NOTE: Same issue w ED doctors in landmark study

• k value=0.93 ( interobserver agreement)- Paramedics’interpretation of the rule good- Difficulty w ‘dangerous mechanisms’ part of

the rule

Effect of CCR

• Reduce no. of immobilisation required to 62%• CCR vs NLR study : Redn to 56% by CCR (Xrays)

Does the paper validate the test?

• Utility of test in our practise• Comparison w true ‘gold standard’• Appropriate spectrum of patients?• Bias• Test reproducibility- In other places outside Canada?- In our SA setting?- Level of EMS training in SA?• Confidence intervals : very wide

Discussion

• 100% sensitivity and NPV using CCR (both paramedic and investigators)

• Good interpretation of the rule by paramedics after a 2-hour tutorial

• Clinical decision rule is reliable : v high k value• Mostly comfortable applying the rule• Application of CCR can potentially reduce no.

Of immobisations by 40%----less ED crowding, less unnecessary labour, reduce pt discomfort en-route to hosp

South Africa

Would you be ‘comfortable’ with your EMS providers ‘clearing’ C-spines?