journal of orthopaedic & sports physical therapy | volume 44 | number 2 | february 2014 | 85

[ clinical commentary ]

Decision making in health care is increasingly informed by the incorporation of knowledge considered within a probabilistic framework.23,29 New information derived from the history, physical examination, and other investigations is used to revise

prior beliefs about the likelihood of a given diagnosis or outcome by a magnitude proportional to the relative strength of that information.10,14 The application of such methods, particularly within a diagnostic

context, has been termed probabilis-tic reasoning.11,31 Within a probabilistic framework, perfect predictions are not anticipated and error is knowingly ac-cepted.12 The goal of probabilistic reason-ing is therefore not to predict outcomes with certainty but, rather, to generate predictions that are more often “less

wrong” than those generated by other methods.

Prior beliefs (prevalence of a diagnosis or outcome), also known as “pretest prob-ability,” and new information (outcome of test with known sensitivity and specific-ity) may be mathematically integrated to produce the quantified probability of a

given diagnosis or outcome (known as a “posterior probability” or “posttest prob-ability”) using a well-known application of Bayes’ theorem.2

(1) P (Diagnosis or Outcome) =

Prevalence ≈ SensitivityPrevalence ≈ Sensitivity +

(1 – Prevalence) ≈ (1 – Specificity)

Clinical prediction rules (CPRs) are a common application of probabilistic reasoning in health care and function to produce estimates of the likelihood of a target diagnosis, prognosis, or treatment outcome that, in turn, inform clinical decision making.16,25 For example, Hicks et al17 derived a CPR that functions to identify patients experiencing low back pain with a higher likelihood of achieving success from an 8-week stabilization-ex-ercise program. They found that when 3 or more factors were present at baseline, including a positive prone instability test, aberrant movements, average straight leg raise greater than 91°, and age greater than 40 years, the probability of achiev-ing a successful treatment outcome in-creased from 33% to 67%. A second example from the physical therapy lit-erature is a CPR designed to identify pa-tients with low back pain more likely to benefit from lumbopelvic thrust manip-ulation. Flynn et al13 identified that the probability of success from this interven-tion increased from 45% to 95% when 4 or more factors were present, including

TT SYNOPSIS: Decision making in physical therapy is increasingly informed by evidence in the form of probabilities. Prior beliefs concerning diagnoses, prognoses, and treatment effects are quantitatively revised by the integration of new information derived from the history, physical examination, and other investigations in a well-recognized application of Bayes’ theorem. Clinical prediction rule development studies commonly employ such methodology to produce quantified estimates of the likelihood of patients having certain diagnoses or achieving given outcomes. To date, the physical therapy literature has been limited to the discus-sion and calculation of the point estimate of such probabilities. The degree of precision associated with the construction of posterior probabilities, which requires consideration of both uncertainty associated with pretest probability and uncertainty associated with test accuracy, remains largely unrecognized and unreported. This paper provides an introduction to the calculation of the uncer-

tainty interval, known as a credible interval, around posterior probability estimates. The method for calculating the credible interval is detailed and illustrated with example data from 2 clinical prediction rule development studies. Two relatively quick and simple methods for approximating the credible interval are also outlined. It is anticipated that knowledge of the credible interval will have practical implications for the incorporation of probabilistic evidence in clinical practice. Consis-tent with reporting standards for interventional and diagnostic studies, it is equally appropriate that studies reporting posterior probabilities calculate and report the level of precision associated with these point estimates. J Orthop Sports Phys Ther 2014;44(2):85-91. Epub 30 October 2013. doi:10.2519/jospt.2014.4877

TT KEY WORDS: Bayes’ theorem, clinical prediction rules, decision making, probability

1School of Health Sciences, The University of Newcastle, Callaghan, Australia. 2School of Mathematical and Physical Sciences, The University of Newcastle, Callaghan, Australia. The authors certify that they have no affiliations with or financial involvement in any organization or entity with a direct financial interest in the subject matter or materials discussed in the article. Address correspondence to Robin Haskins, School of Health Sciences, The Hunter Building, The University of Newcastle, Callaghan, NSW 2308 Australia. E-mail: Robin.Haskins@newcastle.edu.au T Copyright ©2014 Journal of Orthopaedic & Sports Physical Therapy®

ROBIN HASKINS, BPhty (Hons)1 • PETER G. OSMOTHERLY, PhD1 • FRANK TUYL, PhD2 • DARREN A. RIVETT, PhD1

Uncertainty in Clinical Prediction Rules: The Value of Credible Intervals

44-02 Haskins.indd 85 1/17/2014 5:07:28 PM

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®

Dow

nloa

ded

from

ww

w.jo

spt.o

rg a

t on

Dec

embe

r 1,

201

4. F

or p

erso

nal u

se o

nly.

No

othe

r us

es w

ithou

t per

mis

sion

. C

opyr

ight

© 2

014

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®. A

ll ri

ghts

res

erve

d.

86  |  february 2014  |  volume 44  |  number 2  |  journal of orthopaedic & sports physical therapy

[ clinical commentary ]duration of symptoms less than 16 days, at least 1 hip with more than 35° of in-ternal rotation, lumbar hypomobility, no symptoms below the knee, and a score of less than 19 on the work subscale of the Fear-Avoidance Beliefs Questionnaire.37

Uncertainty in Clinical Prediction: Credible IntervalsTo date, the physical therapy literature has been limited to the discussion and calculation of single values (point esti-mates) of posterior probabilities. When uncertainty is considered, it is frequent-ly limited to test accuracy (sensitivity, specificity, likelihood ratio), with rare consideration of the precision of the pre-test probability. The degree of precision associated with the construction of the posterior probability that is frequently reported in many CPR development studies remains largely unrecognized and unreported. This stands in contrast to the near-uniform reporting of confidence in-tervals around estimates of treatment ef-fect and diagnostic test accuracy seen in the modern scientific literature.7,32 While it is accepted that confidence intervals reported in interventional studies have important implications for clinical prac-tice,34 no such considerations have been given to the need for uncertainty inter-vals for posterior probability estimates published in the physical therapy CPR literature.

The precision of the posterior prob-ability estimate requires consideration of the uncertainty associated with preva-lence, as well as the uncertainty associ-ated with test accuracy.1,4 The resultant uncertainty interval within a Bayesian framework is known as a credible inter-val (CrI) and gives the range in which the “true” likelihood of a given diagnosis or outcome lies for a specified probability level.26 Conceptually, it is the probability of a probability. For instance, the range of values expressed in a 95% CrI has a 95% chance of containing the true probability of a given outcome based on the informa-tion available.

FIGURE 1 illustrates the relationship

between pretest probability and posttest probability for 6 likelihood ratio values. The slope of each curve and, therefore, the influence of a 1% change in pretest probability (x-axis) on posttest probabil-ity (y-axis) is defined by the following formula:

(2) Slope =

Likelihood Ratio[Pretest Probability ≈

(Likelihood Ratio – 1) + 1]2

Where the slope of this illustrated rela-tionship is steep, small variations in the pretest probability have a large influence on the posttest probability. Conversely, where the slope is relatively flat, con-sideration of uncertainty of the pretest probability will only have a relatively small impact on the posterior probabil-ity uncertainty interval.4 The degree to which uncertainty in the likelihood ratio impacts the posterior probability is de-pendent on the magnitude of the pretest probability. Therefore, incorporation of uncertainty of both the pretest probabil-

ity and the likelihood ratio is required to calculate the degree of uncertainty of the posterior probability.

Calculating the CrIAn appropriate method to calculate the CrI for a posterior probability estimate of a binary outcome is the objective Bayesian method using Monte Carlo simulation. This method takes into con-sideration uncertainty related to both pretest probability and test accuracy. Mossman and Berger26 identified this approach as having superior performance properties compared to other methods of uncertainty-interval calculation. De-tailed below is a step-by-step guide as to how clinicians and researchers can perform this calculation using Microsoft Excel 2010 (Microsoft Corporation, Red-mond, WA) and the statistical freeware R (http://www.r-project.org/). Each calcu-lation step is illustrated using 2 separate examples from the physical therapy CPR literature. Of note, we have substituted the use of the Jeffreys prior used in Moss-man and Berger’s26 original method with

0%

20%

0% 20% 40% 60% 80% 100%

40%

60%

80%

100%

Post

test

Pro

babi

lity

Pretest Probability

+LR = 50

+LR = 10

+LR = 2

–LR = 0.5

–LR = 0.1

–LR = 0.02

FIGURE 1. The relationship between pretest probability and posttest probability for specified likelihood ratios. The slope of each curve illustrates the degree to which variation in the pretest probability influences the posttest probability. Abbreviation: LR, likelihood ratio.

44-02 Haskins.indd 86 1/17/2014 5:07:29 PM

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®

Dow

nloa

ded

from

ww

w.jo

spt.o

rg a

t on

Dec

embe

r 1,

201

4. F

or p

erso

nal u

se o

nly.

No

othe

r us

es w

ithou

t per

mis

sion

. C

opyr

ight

© 2

014

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®. A

ll ri

ghts

res

erve

d.

journal of orthopaedic & sports physical therapy | volume 44 | number 2 | february 2014 | 87

the uniform Bayes–Laplace prior, due to its additional ability to perform well in instances where there are no false posi-tives.36 There are essentially 6 steps.Step 1 Identify the numerator (x) and denominator (n) for prevalence (P0), sensitivity (P1), and 1 – specificity (P2). In many circumstances, it may be appropri-ate to source prevalence and test accuracy data from separate studies. In the case of a single study, these data may often be de-rived from a standard 2-by-2 contingency table (TABLE 1). If a contingency table has not been provided in the published pa-per, it can sometimes be calculated and constructed by the reader, as in the 2 ex-amples below.

• Example 1. Based on the published results of Hicks et al17 for determining the likelihood of success for a stabili-zation exercise program for low back pain for when 3 or more variables were present at baseline, a 2-by-2 con-tingency table may be derived (TABLE

2). The following calculations may be derived from this table:

P0 = x0/n0 = 18/54 (33%)P1 = x1/n1 = 10/18 (56%)P2 = x2/n2 = 5/36 (14%)

• Example 2. Using the results of Flynn et al13 for identifying the likelihood of treatment success from lumbopelvic

manipulation for low back pain for when 4 or more variables are present, a 2-by-2 contingency table may be derived (TABLE 3) that enables the fol-lowing calculations of prevalence (P0), sensitivity (P1), and 1 – specificity (P2):

P0 = x0/n0 = 32/71 (45%)P1 = x1/n1 = 20/32 (63%)

P2 = x2/n2 = 1/39 (3%)

Step 2 The beta distribution is a type of continuous probability distribution that is employed in Bayesian analysis. The shape parameters a and b that will be used to construct beta distributions (β[a, b]) for prevalence, sensitivity, and 1 – specificity are calculated using the fol-lowing: β(a, b) = β(xi + 1, ni – xi +1).• Example 1. Continuing our example

for stabilization exercises for low back pain:

Prevalence: β(a, b) = β(18 + 1, 54 – 18 + 1) = β(19, 37)

Sensitivity: β(a, b) = β(10 + 1, 18 – 10 + 1) = β(11, 9)

1 – specificity: β(a, b) = β(5 + 1, 36 – 5 + 1) = β(6, 32)

• Example 2. Using the data for lum-bopelvic thrust manipulation for low back pain:

Prevalence: β(a, b) = β(32 + 1, 71 – 32 + 1) = β(33, 40)

Sensitivity: β(a, b) = β(20 + 1, 32 – 20 + 1) = β(21, 13)

1 – specificity: β(a, b) = β(1 + 1, 39 – 1 + 1) = β(2, 39)

Step 3 Use a random number genera-tor to select a large number of values (N) (Mossman and Berger26 suggest 10 000) from each beta distribution. This can be performed using the statistical freeware R, with the following command to fa-cilitate simple importing into Microsoft Excel (replace a and b with the relevant shape parameters calculated in step 2): write.csv(rbeta(10000, a, b)).Step 4 Combine each series of randomly

TABLE 1Two-by-Two Table to Obtain Prevalence,

Sensitivity, and 1 – Specificity*

*Prevalence = (A + C)/(A + B + C + D). Sensitivity = A/(A + C). 1 – specificity = B/(B + D).

Outcome Present Outcome Not Present

Test positive A B

Test negative C D

TABLE 2Two-by-Two Contingency Table Derived

From the Data by Hicks et al17

Abbreviation: CPR, clinical prediction rule.*Percentage change in baseline and 8-week Oswestry Low Back Pain Disability Questionnaire score.†Age less than 40 years, average straight leg raise greater than 91°, aberrant movement present, posi-tive prone instability test.

Treatment Successful (50% or Greater Improvement*)

Treatment Not Successful (Less Than 50% Improvement)

CPR positive (3 or more variables present†) 10 5

CPR negative (fewer than 3 variables present) 8 31

TABLE 3Two-by-Two Contingency Table Derived

From the Data by Flynn et al13

Abbreviation: CPR, clinical prediction rule.*Percentage change in Oswestry Low Back Pain Disability Questionnaire score over 3 sessions.†Symptom duration less than 16 days, at least 1 hip with greater than 35° of internal rotation, hypo-mobility with lumbar spring testing, no symptoms distal to the knee, Fear-Avoidance Beliefs Question-naire work subscale score less than 19.

Treatment Successful (Greater Than 50% Improvement*)

Treatment Not Successful (50% or Less Improvement)

CPR positive (4 or more variables present†) 20 1

CPR negative (fewer than 4 variables present) 12 38

44-02 Haskins.indd 87 1/17/2014 5:07:31 PM

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®

Dow

nloa

ded

from

ww

w.jo

spt.o

rg a

t on

Dec

embe

r 1,

201

4. F

or p

erso

nal u

se o

nly.

No

othe

r us

es w

ithou

t per

mis

sion

. C

opyr

ight

© 2

014

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®. A

ll ri

ghts

res

erve

d.

88  |  february 2014  |  volume 44  |  number 2  |  journal of orthopaedic & sports physical therapy

[ clinical commentary ]drawn values for prevalence (P0), sensi-tivity (P1), and 1 – specificity (P2) using the following formula:

P0 ≈ P1

P0 ≈ P1 + (1 – P0) ≈ P2

(3)

Note that using the point estimates of P0, P1, and P2 in the above will pro-vide the point estimate of the posterior probability.• Example 1. For the stabilization-exer-

cise example:

0.33 ≈ 0.560.33 ≈ 0.56 + (1 – 0.33) ≈ 0.14

= 67%

• Example 2: for the lumbopelvic thrust manipulation example:

0.45 ≈ 0.630.45 ≈ 0.63 + (1 – 0.45) ≈ 0.03

= 95%

Step 5 Next, sort the resultant array of values in ascending order.Step 6 To determine the lower and upper boundaries of a 2-tailed 95% CrI, find the values corresponding to N × 0.025 and N × 0.975, respectively, in the sorted array.• Example 1. For the stabilization-exer-

cise example, the 95% CrI is 41% to 85%.

• Example 2. For the lumbopelvic thrust manipulation example, the 95% CrI is 77% to 99%.Thus, for the stabilization-exercise

example data on the likelihood of suc-cess of this intervention for low back pain, the corresponding 2-tailed 95% CrI is 41% to 85%. This means that, given the data, we can be 95% certain that the true probability of success from this exercise program for a patient pre-senting with 3 or more of the identified baseline predictors is between 41% and 85%. For the lumbopelvic thrust ma-nipulation CPR example, the 95% CrI corresponding to positive status on this rule is 77% to 99%.

For those more experienced in using R, steps 3 to 6 of the method outlined above can be performed entirely within R using the syntax commands listed be-low. Computationally, this method is very

quick and permits the use of an extremely large value for N (eg, 1 million) to pro-duce even more precise estimates of the posterior uncertainty interval, as the cal-culation remains in vector form and does not require exporting to a spreadsheet.

N = 1 000 000P0 = rbeta(N, x0 + 1, n0 – x0 + 1)P1 = rbeta(N, x1 + 1, n1 – x1 + 1)P2 = rbeta(N, x2 + 1, n2 – x2 + 1)

Post = P0 × P1/(P0 × P1 + (1 – P0) × P2)Quantile(Post, c(0.025, 0.975))

FIGURE 2 illustrates the appropriate syntax commands and output for this method in R using the data from the CPR examples provided above.

Quick and Simple ApproximationsIn situations where it may be appropri-ate to gather prevalence, sensitivity, and specificity data from within a single study (this excludes case-control designs), 2 methods may provide quick and relatively simple approximations of the uncertainty interval of the posterior probability that may be sufficiently accurate for clinical purposes in the majority of instances.

Tuyl35 proposed that adequate inter-vals may be constructed by using the beta(true positive + 1, false positive + 1) distribution. To calculate the 95% inter-val using this method, the following for-mulae may be used in Microsoft Excel:

Lower boundary: =beta.inv(0.025, true positive + 1, false positive + 1)

Upper boundary: =beta.inv(0.975, true positive + 1, false positive + 1)

A useful adjunct to this method is that the mode of this distribution will corre-spond to the point estimate of the posteri-or probability. The intervals generated by this method for both the stabilization-ex-ercise example data and the lumbopelvic thrust manipulation example data are al-most identical (less than 1% different) to those generated by the objective Bayesian method using Monte Carlo simulation.

In the instance of zero false positives, a 1-sided interval appears preferable, such that the point estimate (100%) will be in-cluded within the uncertainty interval. To construct a 1-tailed 95% CrI in Microsoft Excel in the case of zero false positives, the following formulae may be applied:

Lower boundary: =beta.inv(0.05, True Positive + 1, 1)

Upper boundary: =1

An alternative method is to calculate the binomial proportion confidence in-terval of the positive predictive value: [True Positives/(True Positives + False Positives)]. The following command in the statistical freeware R will produce the relevant 95% interval using the Clopper-Pearson method9:

FIGURE 2. R syntax commands and output for the calculation of the posterior probability 95% credible interval using the objective Bayesian method with Monte Carlo simulation. (A) Stabilization-exercise CPR example data. (B) Lumbopelvic thrust manipulation CPR example data. Abbreviation: CPR, clinical prediction rule.

44-02 Haskins.indd 88 1/17/2014 5:07:32 PM

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®

Dow

nloa

ded

from

ww

w.jo

spt.o

rg a

t on

Dec

embe

r 1,

201

4. F

or p

erso

nal u

se o

nly.

No

othe

r us

es w

ithou

t per

mis

sion

. C

opyr

ight

© 2

014

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®. A

ll ri

ghts

res

erve

d.

journal of orthopaedic & sports physical therapy | volume 44 | number 2 | february 2014 | 89

binom.test(True Positives, True Positives + False Positives)

This method produces uncertainty in-tervals that tend to be more conservative than the aforementioned methods28 but may nevertheless be adequate for clini-cal decision-making purposes. For the stabilization-exercise example data, this method provides an interval of 38% to 88%, and for the thrust manipulation ex-ample data, an interval of 76% to 100%.

A Brief Discussion on Selected Alternative MethodsIt may seem intuitive to many to calculate the boundaries of the uncertainty inter-val of the posterior probability by com-bining the lower boundary of the pretest probability confidence interval with the lower boundary of the likelihood ratio confidence interval, and repeating this procedure for the upper boundaries. This method, however, produces uncertainty intervals that are generally much more conservative than the aforementioned procedures. For the stabilization-exercise example data, this method produces an interval of 30% to 90%, and for the lum-bopelvic thrust manipulation example data, an interval of 63% to 100%. Given the computational steps involved in com-pleting this method, it is also unlikely to be considered simpler than the approxi-mation methods previously outlined.

As already discussed, in some instanc-es, uncertainty in the pretest probability will only have a very small influence on the uncertainty of the posttest probabil-ity. In such cases, simply combining the point estimate of the pretest probability with the upper and lower boundaries of the likelihood ratio confidence inter-val will produce an uncertainty interval similar to that produced by the methods already outlined. For the stabilization-exercise example data, this method pro-duces an interval of 45% to 83%, and for the lumbopelvic thrust manipulation ex-ample data, an interval of 74% to 99%. This method, however, may only give a suitable approximation when small

variations in the pretest probability do not greatly influence the posttest prob-ability across any portion of the pretest probability uncertainty interval. The in-fluence of the pretest probability on the postprobability at a given likelihood ratio value may be calculated at its upper and lower uncertainty intervals using formula 2 presented above. Alternatively, it can be approximated by considering the slope of the corresponding parts of the likelihood ratio curves presented in FIGURE 1. Given the calculations required to check the appropriateness of this approximation method, it will be quicker and simpler in most cases to use the methods previously recommended.

Future DirectionsThe additional knowledge of the CrI surrounding a posterior probability point estimate has important implica-tions concerning the appropriate clini-cal application of probabilistic evidence into practice. Rather than sole reliance on the point estimate, consideration of its precision, as highlighted by the cor-responding CrI, provides substantially greater depth of information, which will assist decision making. Analogous to the use of confidence intervals reported for treatment effect sizes,34 the CrI of a pos-terior probability may be used to inform clinical decision making by considering its position relative to a “threshold level of certainty”14 required by a clinician to make a decision, such as commenc-ing a particular treatment program or confirming or negating a diagnostic hy-pothesis. Such thresholds are dependent on the relative risks and benefits associ-ated with that decision. In the context of treatment decision making, a clinician may have a relatively low threshold of certainty required for a treatment that is low cost and low risk, and may have a higher threshold of certainty required for a treatment that is associated with a higher risk of an adverse outcome and is more expensive and time consuming.

In the stabilization-exercise CPR ex-ample provided, it is plausible that the

lower boundary of the uncertainty in-terval (41%) may be below a “treatment threshold”14 many clinicians and patients would consider sufficient for an 8-week treatment program that requires 16 su-pervised sessions and daily home exer-cises. Within a probabilistic framework, further confirmatory evidence from the history and physical examination would be required for clinicians and patients to have confidence in the benefit that may be achieved from this intervention. To their credit, Hicks et al17 explicitly stated that the results of their study about the likelihood of success from a program of stabilization exercises represent the preliminary step in the development of a CPR, and they have not recommended that the derived tool be implemented in clinical practice. We have used the data from their derivation study to illustrate how clinical decisions informed by pos-terior probabilities may be influenced by the additional knowledge of the CrI.

In our second CPR example, concern-ing the probability of treatment success from lumbopelvic thrust manipulation, the published results of Flynn et al13 en-able the calculation of a CrI of 77% to 99%. In contrast to the stabilization-exercise program example, it is plau-sible that many clinicians and patients may perceive that the lower boundary of this uncertainty interval (77%) is above a suitable treatment threshold14 for this intervention, given its relatively low cost, short time frame, and low risk of serious adverse events.33

The primary method of calculating the CrI outlined in this paper (objective Bayesian method using Monte Carlo sim-ulation) enables the incorporation of data from more than 1 study. Given that sev-eral independent studies are required in the development of a CPR,25 meta-anal-ysis of such data may be helpful in the construction of more precise estimates of the posterior probability interval. For example, a recent systematic review5 of a CPR designed to help identify patients at risk of falling used a meta-analysis of the included studies to calculate more

44-02 Haskins.indd 89 1/17/2014 5:07:34 PM

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®

Dow

nloa

ded

from

ww

w.jo

spt.o

rg a

t on

Dec

embe

r 1,

201

4. F

or p

erso

nal u

se o

nly.

No

othe

r us

es w

ithou

t per

mis

sion

. C

opyr

ight

© 2

014

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®. A

ll ri

ghts

res

erve

d.

90  |  february 2014  |  volume 44  |  number 2  |  journal of orthopaedic & sports physical therapy

[ clinical commentary ]18. Irwig L, Tosteson AN, Gatsonis C, et al. Guide-

lines for meta-analyses evaluating diagnostic tests. Ann Intern Med. 1994;120:667-676. http://dx.doi.org/10.7326/0003-4819-120-8-199404150-00008

19. Jones CM, Athanasiou T. Diagnostic accuracy meta-analysis: review of an important tool in radiological research and decision making. Br J Radiol. 2009;82:441-446. http://dx.doi.org/10.1259/bjr/28171084

20. Kamper SJ, Maher CG, Hancock MJ, Koes BW, Croft PR, Hay E. Treatment-based subgroups of low back pain: a guide to appraisal of research studies and a summary of current evidence. Best Pract Res Clin Rheumatol. 2010;24:181-191. http://dx.doi.org/10.1016/j.berh.2009.11.003

21. Kent P, Hancock M, Petersen DH, Mjøsund HL. Clinimetrics corner: choosing appropriate study designs for particular questions about treatment subgroups. J Man Manip Ther. 2010;18:147-152. http://dx.doi.org/10.1179/106698110X12640740712419

22. Kent P, Keating JL, Leboeuf-Yde C. Research methods for subgrouping low back pain. BMC Med Res Methodol. 2010;10:62. http://dx.doi.org/10.1186/1471-2288-10-62

23. Ledley RS, Lusted LB. Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason. Science. 1959;130:9-21.

24. Macaskill P, Gatsonis C, Deeks JJ, Harbord RM, Takwoingi Y. Chapter 10: analysing and present-ing results. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Oxford, UK: The Cochrane Collaboration; 2010.

25. McGinn T, Wyer P, Wisnivesky J, et al. Advanced topics in diagnosis: clinical prediction rules. In: Guyatt G, Rennie D, Meade MO, Cook DJ, eds. Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2nd ed. New York, NY: McGraw-Hill Medical; 2008:491-505.

26. Mossman D, Berger JO. Intervals for posttest probabilities: a comparison of 5 methods. Med Decis Making. 2001;21:498-507.

27. Nee RJ, Coppieters MW. Interpreting research on clinical prediction rules for physiotherapy treatments. Man Ther. 2011;16:105-108. http://dx.doi.org/10.1016/j.math.2010.10.005

28. Newcombe RG. Two-sided confidence intervals for the single proportion: com-parison of seven methods. Stat Med. 1998;17:857-872. http://dx.doi.org/10.1002/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E

29. O’Connor GT, Sox HC, Jr. Bayesian reasoning in medicine: the contributions of Lee B. Lusted, MD. Med Decis Making. 1991;11:107-111.

30. Reitsma JB, Rutjes AWS, Whiting P, Vlassov VV, Leeflang MMG, Deeks JJ. Chapter 9: assessing methodological quality. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Handbook for

REFERENCES

1. Baron JA. Uncertainty in Bayes. Med Decis Mak-ing. 1994;14:46-51.

2. Bayes TR. An essay towards solving a prob-lem in the doctrine of chances. Phil Trans. 1763;53:370-418. http://dx.doi.org/10.1098/rstl.1763.0053

3. Beattie P, Nelson R. Clinical prediction rules:

what are they and what do they tell us? Aust J Physiother. 2006;52:157-163.

4. Bianchi MT, Alexander BM, Cash SS. Incorporat-ing uncertainty into medical decision making: an approach to unexpected test results. Med Decis Making. 2009;29:116-124. http://dx.doi.org/10.1177/0272989X08323620

5. Billington J, Fahey T, Galvin R. Diagnostic ac-curacy of the STRATIFY clinical prediction rule for falls: a systematic review and meta-analysis. BMC Fam Pract. 2012;13:76. http://dx.doi.org/10.1186/1471-2296-13-76

6. Bossuyt PM, Leeflang MM. Chapter 6: develop-ing criteria for including studies. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Hand-book for Systematic Reviews of Diagnostic Test Accuracy. Oxford, UK: The Cochrane Collabora-tion; 2008.

7. Bossuyt PM, Reitsma JB, Bruns DE, et al. To-wards complete and accurate reporting of stud-ies of diagnostic accuracy: the STARD initiative. BMJ. 2003;326:41-44.

8. Childs JD, Cleland JA. Development and ap-plication of clinical prediction rules to improve decision making in physical therapist practice. Phys Ther. 2006;86:122-131.

9. Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika. 1934;26:404-413. http://dx.doi.org/10.1093/biomet/26.4.404

10. Davidson M. The interpretation of diagnostic tests: a primer for physiotherapists. Aust J Physiother. 2002;48:227-232.

11. Doust J. Diagnosis in general practice. Using probabilistic reasoning. BMJ. 2009;339:b3823.

12. Einhorn HJ. Accepting error to make less error. J Pers Assess. 1986;50:387-395. http://dx.doi.org/10.1207/s15327752jpa5003_8

13. Flynn T, Fritz J, Whitman J, et al. A clinical prediction rule for classifying patients with low back pain who demonstrate short-term im-provement with spinal manipulation. Spine (Ph-ila Pa 1976). 2002;27:2835-2843. http://dx.doi.org/10.1097/01.BRS.0000035681.33747.8D

14. Fritz JM, Wainner RS. Examining diagnostic tests: an evidence-based perspective. Phys Ther. 2001;81:1546-1564.

15. Hancock M, Herbert RD, Maher CG. A guide to interpretation of studies investigating sub-groups of responders to physical therapy inter-ventions. Phys Ther. 2009;89:698-704. http://dx.doi.org/10.2522/ptj.20080351

16. Haskins R, Rivett DA, Osmotherly PG. Clinical prediction rules in the physiotherapy manage-ment of low back pain: a systematic review. Man Ther. 2012;17:9-21. http://dx.doi.org/10.1016/j.math.2011.05.001

17. Hicks GE, Fritz JM, Delitto A, McGill SM. Preliminary development of a clinical predic-tion rule for determining which patients with low back pain will respond to a stabilization exercise program. Arch Phys Med Rehabil. 2005;86:1753-1762. http://dx.doi.org/10.1016/j.apmr.2005.03.033

precise estimates of the tool’s sensitiv-ity and specificity. These data were then used to help calculate more precise esti-mates of the posttest probability of a fall in patients who were either positive or negative on that CPR. To the best of our knowledge, published guidelines specific to the meta-analysis of CPR studies do not yet exist; however, many consider-ations concerning the appropriateness of pooling CPR accuracy data may be plausibly generalized from guidelines on the meta-analysis of diagnostic tests. Such considerations include the meth-odological rigor of included studies, as well as between-study variations in study design, study participants, CPR applica-tion, and the assessment of the reference standard.6,18,19,24,30

CONCLUSION

This commentary focuses on the precision of posterior probability es-timates in CPR development stud-

ies. This is a necessary, but not sufficient, consideration in the application of such evidence into clinical practice. Other fac-tors beyond the scope of this commentary, including study design, methodological quality, validation, and impact analysis, require equally due consideration and have been well described in the existing physical therapy literature.3,8,15,20-22,27 Con-sistent with reporting standards for in-terventional32 and diagnostic7 studies, we believe that it is equally appropriate that studies reporting posterior probabilities, as commonly practiced in CPR develop-ment studies, calculate and report the level of precision associated with these point estimates. t

44-02 Haskins.indd 90 1/17/2014 5:07:34 PM

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®

Dow

nloa

ded

from

ww

w.jo

spt.o

rg a

t on

Dec

embe

r 1,

201

4. F

or p

erso

nal u

se o

nly.

No

othe

r us

es w

ithou

t per

mis

sion

. C

opyr

ight

© 2

014

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®. A

ll ri

ghts

res

erve

d.

journal of orthopaedic & sports physical therapy | volume 44 | number 2 | february 2014 | 91

@ MORE INFORMATIONWWW.JOSPT.ORG

Systematic Reviews of Diagnostic Test Accu-racy. Oxford, UK: The Cochrane Collaboration; 2009.

31. Richardson WS, Wilson MC. Chapter 14: the pro-cess of diagnosis. In: Guyatt G, Rennie D, Meade MO, Cook DJ, eds. Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clini-cal Practice. 2nd ed. New York, NY: McGraw-Hill Medical; 2008:399-406.

32. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for re-porting parallel group randomized trials. Ann Intern Med. 2010;152:726-732. http://dx.doi.org/10.7326/0003-4819-152-11-201006010-

00232 33. Shekelle PG, Adams AH, Chassin MR, Hurwitz

EL, Brook RH. Spinal manipulation for low-back pain. Ann Intern Med. 1992;117:590-598.

34. Stratford PW. The added value of confidence intervals. Phys Ther. 2010;90:333-335. http://dx.doi.org/10.2522/ptj.2010.90.3.333

35. Tuyl F. Diagnostic testing: a simple approximate interval for the positive predictive value. Pro-ceedings of the Third Annual ASEARC Research Conference. Newcastle, Australia: Applied Sta-tistics Education and Research Collaboration; 2009.

36. Tuyl F, Gerlach R, Mengersen K. A com-

parison of Bayes–Laplace, Jeffreys, and other priors: the case of zero events. Am Statistician. 2008;62:40-44. http://dx.doi.org/10.1198/000313008X267839

37. Waddell G, Newton M, Henderson I, Somerville D, Main CJ. A Fear-Avoidance Beliefs Question-naire (FABQ) and the role of fear-avoidance beliefs in chronic low back pain and disability. Pain. 1993;52:157-168.

GO GREEN By Opting Out of the Print Journal

JOSPT subscribers and APTA members of the Orthopaedic and Sports Physical Therapy Sections can help the environment by “opting out” of receiving JOSPT in print each month as follows. If you are:

• A JOSPT subscriber: Email your request to jospt@jospt.org or call the JOSPT o�ce toll-free at 1-877-766-3450 and provide your name and subscriber number.

• APTA Orthopaedic or Sports Section member: Go to http://www.apta.org/, log in, and select My Profile. Next click on Email Management/GoGreen. Toward the bottom of the list, you will find the Publications options and may opt out of receiving the print JOSPT. Please save this preference.

Subscribers and members alike will continue to have access to JOSPT online and can retrieve current and archived issues anytime and anywhere you have Internet access.

44-02 Haskins.indd 91 1/17/2014 5:07:35 PM

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®

Dow

nloa

ded

from

ww

w.jo

spt.o

rg a

t on

Dec

embe

r 1,

201

4. F

or p

erso

nal u

se o

nly.

No

othe

r us

es w

ithou

t per

mis

sion

. C

opyr

ight

© 2

014

Jour

nal o

f O

rtho

paed

ic &

Spo

rts

Phys

ical

The

rapy

®. A

ll ri

ghts

res

erve

d.