unsafety: making no mockery of honest ad-hockery janet wittes statistics collaborative asa/fda 2005
TRANSCRIPT
![Page 1: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/1.jpg)
Unsafety: Making no mockery of honest ad-hockery
Janet WittesStatistics Collaborative
ASA/FDA 2005
![Page 2: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/2.jpg)
Topic du jour
![Page 3: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/3.jpg)
Sleight of tongue
Game: Remove safety and add virtue in two easy steps
D S M C_ _ __ _ _ _
![Page 4: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/4.jpg)
Sleight of tongue
Game: Remove safety and add virtue in two easy steps
D S M CD M C_ _ _ _
![Page 5: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/5.jpg)
Sleight of tongue
Game: Remove safety and add virtue in two easy steps
D S M CD M CI D M C
![Page 6: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/6.jpg)
Efficacy
Stop for “overwhelming” efficacy =0.05; power = 90%; four looks Probability of stopping early>70% Early stop: estimate pulls toward
null
![Page 7: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/7.jpg)
Our current approaches
Prespecify Classify precisely Give lots of data Rely on mechanism
![Page 8: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/8.jpg)
Divide and (un)conquer (e.g. Neuropathy)
Event T C
Neuropathic pain 1 0
Neuropathy 1 0
Neuropathy NOS 5 2
Neuropathy peripheral 2 0
…
…
…
![Page 9: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/9.jpg)
e.g. Neuropathy
Event T C
Anosmia
…..
Autonomic neuropathy 1 0
…
Cranial neuropathy 2 1
…
…
![Page 10: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/10.jpg)
e.g. Neuropathy
Event T C
…
Parathesia 3 2
Parathesia NOS 4 0
Parathesia other 0 1
…
Peripheral motor neuropathy 6 0
Peripheral sensory neuropathy 3 2
![Page 11: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/11.jpg)
Other examples
Heart failure• Separate near synonyms• Allocate to heart and lung
![Page 12: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/12.jpg)
Other examples
Heart failure• Separate near synonyms• Allocate to heart and lung
Bleeding: distribute over body systems
![Page 13: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/13.jpg)
Consider mechanism
If you don’t get the drug, you can’t react to it• Eschew ITT: safety population• Modified Daley’s Rule: censor early and
often• Don’t collect extraneous information
Appeal to statistical conservatism
![Page 14: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/14.jpg)
Populations
ITT mITT Safety: one dose of study med ATP Etc.
![Page 15: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/15.jpg)
e.g., Vioxx- short follow-up
![Page 16: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/16.jpg)
Through 36 months
![Page 17: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/17.jpg)
With denominators
Bresalier et al., NEJM, Feb 2005
![Page 18: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/18.jpg)
The denominators
Mo Rx Po0 1287 129912 1129 119518 1057 115624 938 104230 896 100136 727 835
![Page 19: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/19.jpg)
No data dredging
We test hypotheses Too many type 1 errors if we
dredge
![Page 20: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/20.jpg)
What we typically present
Current behavior• Same tables for interim & final
analyses• Long complete listings
What we should do• Interim data are different from final• Presenting too much dulls the mind
![Page 21: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/21.jpg)
Sentinel events
Single event (e.g., death in a vaccine trial)
Several events (e.g., 3 retinal thromboses)
Sentinel event rate (e.g., WHI)
Lachenbruch and Wittes, SIM, to appear
![Page 22: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/22.jpg)
How to handle
1. Identify sentinel event2. Establish stat’l method for future
events Need reasonable power Type I error rate > 1-sided 0.025
![Page 23: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/23.jpg)
Methods
Individual• Number of non-events until the k’th
eventNegative binomial SPRT
• Time to the next (or k’th) event (-dist’n) Rate: event rate in future patients
NormalPoisson
![Page 24: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/24.jpg)
Pitfalls
Time is subtle Power is low Censoring is tricky
![Page 25: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/25.jpg)
Pull-up
Safety hypothesis: • E.g., Cox-2 leads to 2 fold increase in
MI etc.• Design: stop early if you reject• Estimated relative risk must be pulled
toward 2 Insight from Joe Heyse
![Page 26: Unsafety: Making no mockery of honest ad-hockery Janet Wittes Statistics Collaborative ASA/FDA 2005](https://reader036.vdocument.in/reader036/viewer/2022062511/5514bcfc55034640138b569d/html5/thumbnails/26.jpg)
The option
Respect PI: Adjudicate adverse events
Precision: Reclassify, reorganize Mechanism: Be an empiricist Dredging: Use sentinel events