the good, the bad & the ugly of ems research. definitions research: careful, systematic, patient...
TRANSCRIPT
DefinitionsDefinitions
Research: careful, systematic, patient study and investigation in some field of knowledge, undertaken to discover or establish facts or principles.
Research: careful, systematic, patient study and investigation in some field of knowledge, undertaken to discover or establish facts or principles.
DefinitionsDefinitions
Science:The state or fact of knowing; knowledge.
Systematized knowledge derived from observation, study, and experimentation carried on in order to determine the nature or principles of what is being studied.
Science:The state or fact of knowing; knowledge.
Systematized knowledge derived from observation, study, and experimentation carried on in order to determine the nature or principles of what is being studied.
Is EMS Art or Science?Is EMS Art or Science?
The knowledge of EMS is science.
The way it is applied is art.
Excellent EMTs know the science of EMS and use the art of EMS to apply the science.
The knowledge of EMS is science.
The way it is applied is art.
Excellent EMTs know the science of EMS and use the art of EMS to apply the science.
Art or Science?Art or Science?
Some health practices are more about art than science.
Chiropractic, for example, has little science and a lot of art.
Some health practices are more about art than science.
Chiropractic, for example, has little science and a lot of art.
Art or Science?Art or Science?
Neurology today has a great deal of science and little art.
Neurology today has a great deal of science and little art.
Art or Science?Art or Science?
When you have a life threatening illness or injury, would you seek out a health care provider whose practice was more art or more science?
When you have a life threatening illness or injury, would you seek out a health care provider whose practice was more art or more science?
Art or Science?Art or Science?
EMS is the safety net of society.
We are often the first to provide care to the injured and the infirm.
Our practices must be based on science.
EMS is the safety net of society.
We are often the first to provide care to the injured and the infirm.
Our practices must be based on science.
ResearchResearch
Most medical research is based on the application of the scientific method.
Most medical research is based on the application of the scientific method.
The Scientific MethodThe Scientific Method
The scientific method is the process by which scientists, collectively and over time, endeavor to construct an accurate (that is, reliable, consistent and non-arbitrary) representation of the world.
The scientific method is the process by which scientists, collectively and over time, endeavor to construct an accurate (that is, reliable, consistent and non-arbitrary) representation of the world.
The Scientific MethodThe Scientific Method
Steps:1. Observe some aspect of the universe2. Invent a tentative description, called a hypothesis,
that is consistent with what you observed.3. Use the hypothesis to make predictions.4. Test those predictions by experiments or further
observation and modify the hypothesis in the light of your results.
5. Repeat steps 3 and 4 until there are no discrepancies between theory and experiment and/or observation.
Steps:1. Observe some aspect of the universe2. Invent a tentative description, called a hypothesis,
that is consistent with what you observed.3. Use the hypothesis to make predictions.4. Test those predictions by experiments or further
observation and modify the hypothesis in the light of your results.
5. Repeat steps 3 and 4 until there are no discrepancies between theory and experiment and/or observation.
The Scientific MethodThe Scientific Method
Hypothesis: groundwork, foundation supposition; an unproven theory.
Hypothesis: groundwork, foundation supposition; an unproven theory.
The Scientific MethodThe Scientific Method
The great advantage of the scientific method is that it is unprejudiced:
One does not have to believe a given researcher, one can redo the experiment and determine whether his or her results are true or false. The conclusions will hold irrespective of the state of mind, or the religious persuasion, or the state of consciousness of the investigator and/or the subject of the investigation. Faith, defined as a belief that is not based on logical proof or material evidence, does not determine whether a scientific theory is adopted or discarded.
The great advantage of the scientific method is that it is unprejudiced:
One does not have to believe a given researcher, one can redo the experiment and determine whether his or her results are true or false. The conclusions will hold irrespective of the state of mind, or the religious persuasion, or the state of consciousness of the investigator and/or the subject of the investigation. Faith, defined as a belief that is not based on logical proof or material evidence, does not determine whether a scientific theory is adopted or discarded.
The Scientific MethodThe Scientific Method
A theory is accepted not based on the prestige or convincing powers of the proponent, but on the results obtained through observations and/or experiments which anyone can reproduce.
The results obtained using the scientific method are repeatable.
A theory is accepted not based on the prestige or convincing powers of the proponent, but on the results obtained through observations and/or experiments which anyone can reproduce.
The results obtained using the scientific method are repeatable.
The Scientific MethodThe Scientific Method
There are many types of “pseudoscientific” theories which seem based on a mantle of apparent experimental evidence but that, when examined closely, are nothing but statements of faith.
There are many types of “pseudoscientific” theories which seem based on a mantle of apparent experimental evidence but that, when examined closely, are nothing but statements of faith.
The Scientific MethodThe Scientific Method
“Faith is believing what you know ain’t so.”
Mark Twain, 1894
“Following the Equator”
“Faith is believing what you know ain’t so.”
Mark Twain, 1894
“Following the Equator”
Science versus PseudoscienceScience versus Pseudoscience
How to tell the difference:Anecdotes do not make a science.
Scientific language does not make a science.
Bold statements do not make claims true.
Heresy does not equal correctness.
Reversal of the burden of proof.
Rumors do not equal reality.
How to tell the difference:Anecdotes do not make a science.
Scientific language does not make a science.
Bold statements do not make claims true.
Heresy does not equal correctness.
Reversal of the burden of proof.
Rumors do not equal reality.
Science versus PseudoscienceScience versus Pseudoscience
How to tell the difference:Failures are rationalized.Use of emotive words and false analogies.Ad ignorantiam reasoning (if you can’t disprove a claim, it must be true).Ad hominem or tu quoque statements.Overreliance on authorities.Circular reasoning.Reductio ad absurdum reasoning.
How to tell the difference:Failures are rationalized.Use of emotive words and false analogies.Ad ignorantiam reasoning (if you can’t disprove a claim, it must be true).Ad hominem or tu quoque statements.Overreliance on authorities.Circular reasoning.Reductio ad absurdum reasoning.
The Scientific MethodThe Scientific Method
Enzyte has to work….after all, Bob is smiling!
Enzyte has to work….after all, Bob is smiling!
The Scientific MethodThe Scientific Method
Pseudoscience is still a major part of our culture. Many practitioners resort to pseudoscience and other anecdotal practices.
Pseudoscience is still a major part of our culture. Many practitioners resort to pseudoscience and other anecdotal practices.
The Scientific MethodThe Scientific Method
Typically uses terms and images to appear “scientific”.
Typically uses terms and images to appear “scientific”.
The Scientific MethodThe Scientific Method
Reliance on “names” and anecdotes.
Dr. Greg Cynaumon?Physician?
Psychologist?
Nutritionist?
Chiropractor?
Master’s and Doctorate in psychology from Sierra University (known diploma mill).
Reliance on “names” and anecdotes.
Dr. Greg Cynaumon?Physician?
Psychologist?
Nutritionist?
Chiropractor?
Master’s and Doctorate in psychology from Sierra University (known diploma mill).
The Scientific MethodThe Scientific Method
No clinical evidence CortiSlim works.
No information about contents (natural or herbal).
Dr. Talcott fringe nutritionist with loose affiliations with University of Utah.
One of the largest scams in recent history.
FDA issued a cease and desist order and fined Talbot and Cynamoun.
No clinical evidence CortiSlim works.
No information about contents (natural or herbal).
Dr. Talcott fringe nutritionist with loose affiliations with University of Utah.
One of the largest scams in recent history.
FDA issued a cease and desist order and fined Talbot and Cynamoun.
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
Observation: During a discussion you and your coworkers bring up the idea that there are more psychiatric emergencies when the moon is full.
Hypothesis: Psychiatric emergencies are more common when the lunar cycle is in the full moon phase.
Observation: During a discussion you and your coworkers bring up the idea that there are more psychiatric emergencies when the moon is full.
Hypothesis: Psychiatric emergencies are more common when the lunar cycle is in the full moon phase.
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
Prediction: You predict that there will be more EMS calls for psychiatric emergencies when the moon is full as compared to other times of the month.
Prediction: You predict that there will be more EMS calls for psychiatric emergencies when the moon is full as compared to other times of the month.
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
Testing: You set up an experimental protocol to test your hypothesis.
For this you:Determine you will conduct the study for a year.
Determine when a full moon is scheduled to appear.
Determine that you will declare two days before and two days after the full moon a five-day “full moon period.”
Testing: You set up an experimental protocol to test your hypothesis.
For this you:Determine you will conduct the study for a year.
Determine when a full moon is scheduled to appear.
Determine that you will declare two days before and two days after the full moon a five-day “full moon period.”
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
For this you:Define what a psychiatric emergency will be.
Gather information over the course of the year that includes the date of all psychiatric emergency patients.
When the data has been gathered for the period previously determined, you determine:
1. How many psychiatric patients were there?
2. How many were treated during the full moon periods?
For this you:Define what a psychiatric emergency will be.
Gather information over the course of the year that includes the date of all psychiatric emergency patients.
When the data has been gathered for the period previously determined, you determine:
1. How many psychiatric patients were there?
2. How many were treated during the full moon periods?
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
For this you:Analyze the data based on the number of emergencies that arose during full moon periods and compare to the number of emergencies during the other periods.
Perform any statistical tests necessary to understand the data.
For this you:Analyze the data based on the number of emergencies that arose during full moon periods and compare to the number of emergencies during the other periods.
Perform any statistical tests necessary to understand the data.
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
Results:Results:
Full Moon PeriodDays=65
Other PeriodDays = 300
Total Days = 365
Ψ patients = 10 Ψ patients = 74 Total Ψ patients = 84
Total Patients = 2,105
Total Patients = 9,300
Total Patients = 11,315
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
Results: Your study finds:During full moon period 0.5% of patients were Ψ patients.
During other period 0.8% of patients were Ψ patients.
During the entire study period 0.7% of patients were Ψ patients.
Results: Your study finds:During full moon period 0.5% of patients were Ψ patients.
During other period 0.8% of patients were Ψ patients.
During the entire study period 0.7% of patients were Ψ patients.
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
Results: Your study finds:The incidence of Ψ patients during the full moon period was 0.15 patients per day.
The incidence of Ψ patients during the other moon phases was 0.25 patients per day.
The incidence of Ψ patients overall was 0.23 patients per day.
Results: Your study finds:The incidence of Ψ patients during the full moon period was 0.15 patients per day.
The incidence of Ψ patients during the other moon phases was 0.25 patients per day.
The incidence of Ψ patients overall was 0.23 patients per day.
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
Psych patients are less common during the full moon.
HYPOTHESIS DISSPROVED!
Psych patients are less common during the full moon.
HYPOTHESIS DISSPROVED!
Use of the Scientific Method in EMSUse of the Scientific Method in EMS
Revise hypothesis:New hypothesis: Psychiatric emergencies are no more frequent during full moon lunar phases than during other lunar phases.
HYPOTHESIS ACCURATE.Report findings. Paper should be detailed enough that anybody can follow it and repeat your experiment.
Repeat study to determine whether findings can be repeated.
Revise hypothesis:New hypothesis: Psychiatric emergencies are no more frequent during full moon lunar phases than during other lunar phases.
HYPOTHESIS ACCURATE.Report findings. Paper should be detailed enough that anybody can follow it and repeat your experiment.
Repeat study to determine whether findings can be repeated.
EMS ResearchEMS Research
The more valid a study, the closer it is to the truth!
The more valid a study, the closer it is to the truth!
EMS ResearchEMS Research
Bias: A mental leaning or inclination; partiality; prejudice; bent.
The more a study design adheres to the scientific method, the less chance for bias to affect the outcomes.
Bias: A mental leaning or inclination; partiality; prejudice; bent.
The more a study design adheres to the scientific method, the less chance for bias to affect the outcomes.
EMS ResearchEMS Research
Types of biases:Selection bias
Volunteer bias (tends to favor the treatment group)
Nonrespondent bias
Group not homogeneous
Measurement biasInstrument bias
Insensitive measure bias
Expectation bias
Recall bias (people remember positive events rather than negative events)
Attention bias (aware of participation)
Types of biases:Selection bias
Volunteer bias (tends to favor the treatment group)
Nonrespondent bias
Group not homogeneous
Measurement biasInstrument bias
Insensitive measure bias
Expectation bias
Recall bias (people remember positive events rather than negative events)
Attention bias (aware of participation)
EMS ResearchEMS Research
Types of biases:Intervention bias
Contamination bias
Compliance bias
Withdrawal bias
Proficiency bias
Publication bias
Cherry picking (mainly retrospective studies)
Trashcan bias
Types of biases:Intervention bias
Contamination bias
Compliance bias
Withdrawal bias
Proficiency bias
Publication bias
Cherry picking (mainly retrospective studies)
Trashcan bias
EMS ResearchEMS Research
Avoiding bias:Randomize subjects
Prospective design
Blind subjects and researchers
Contract to publish irregardless of findings
Admit conflicts of interest
Use null hypothesis
Avoiding bias:Randomize subjects
Prospective design
Blind subjects and researchers
Contract to publish irregardless of findings
Admit conflicts of interest
Use null hypothesis
EMS ResearchEMS Research
Bias (sometimes flat deception) has been a common practice in medicine and EMS.
Quackery and bias still permeates many aspects of modern healthcare.
Bias (sometimes flat deception) has been a common practice in medicine and EMS.
Quackery and bias still permeates many aspects of modern healthcare.
EMS ResearchEMS Research
Validity: Whether the study measures what it was supposed to measure.
Validity refers to the appropriateness of the interpretation of the results of a study.
External Validity: The extent and appropriateness of the generalizability of results.
Internal Validity: The basic minimum control, measure, analysis and procedures necessary to make results interpretable.
Validity: Whether the study measures what it was supposed to measure.
Validity refers to the appropriateness of the interpretation of the results of a study.
External Validity: The extent and appropriateness of the generalizability of results.
Internal Validity: The basic minimum control, measure, analysis and procedures necessary to make results interpretable.
EMS ResearchEMS Research
Constant: A characteristic or condition that is the same for all individuals in a study.
Variable:A characteristic that takes on different values or conditions for different individuals.
Dependent Variable: the variable being affected or assumed to be affected my the independent variable.Independent Variables: the variables that affects (or is assumed to affect) the dependent variable under study.Experimental Variable: at least one independent variable being manipulated by the researcher.
Constant: A characteristic or condition that is the same for all individuals in a study.
Variable:A characteristic that takes on different values or conditions for different individuals.
Dependent Variable: the variable being affected or assumed to be affected my the independent variable.Independent Variables: the variables that affects (or is assumed to affect) the dependent variable under study.Experimental Variable: at least one independent variable being manipulated by the researcher.
Levels of EvidenceLevels of Evidence
Center for Evidence-Based Medicine (Oxford)
Ia. Meta-analysis of RCTsIb. One RCT.IIa. Controlled trial without randomisation.IIb. One other type of quasi-experimental study.III. Descriptive studies, such as comparative studies, correlation studies, and case-control studies.IV. Expert committee reports or opinions, or clinical experience of respected authorities or both.
Center for Evidence-Based Medicine (Oxford)
Ia. Meta-analysis of RCTsIb. One RCT.IIa. Controlled trial without randomisation.IIb. One other type of quasi-experimental study.III. Descriptive studies, such as comparative studies, correlation studies, and case-control studies.IV. Expert committee reports or opinions, or clinical experience of respected authorities or both.
Levels of EvidenceLevels of Evidence
American Heart Association1. Positive randomized controlled trials.
2. Neutral randomized controlled trials.
3. Prospective, non-randomized controlled trials.
4. Retrospective, non-randomized controlled trials
5. Case series (no control group)
6. Animal studies
7. Extrapolations
8. Rational conjecture (common sense)
American Heart Association1. Positive randomized controlled trials.
2. Neutral randomized controlled trials.
3. Prospective, non-randomized controlled trials.
4. Retrospective, non-randomized controlled trials
5. Case series (no control group)
6. Animal studies
7. Extrapolations
8. Rational conjecture (common sense)
Study DesignsStudy Designs
Randomized Controlled Trial (RCT):A group of patients are randomized into an experimental group and a control group. These groups are followed up for the variables/outcomes of interest.
RCTs most closely approximate the scientific method and are the most valid of studies.
Randomized Controlled Trial (RCT):A group of patients are randomized into an experimental group and a control group. These groups are followed up for the variables/outcomes of interest.
RCTs most closely approximate the scientific method and are the most valid of studies.
EMS ResearchEMS Research
Houston MAST Study:Constant: All patients with abdominal trauma with hypotension transported by the HFD to Ben Taub Hospital.
Dependent Variable: survival from trauma.
Independent Variables: age, sex, location of injury, transport time, etc.
Experimental Variable: Application or non-application of the MAST.
Houston MAST Study:Constant: All patients with abdominal trauma with hypotension transported by the HFD to Ben Taub Hospital.
Dependent Variable: survival from trauma.
Independent Variables: age, sex, location of injury, transport time, etc.
Experimental Variable: Application or non-application of the MAST.
ExampleExample
Mattox KL, Bickell B, Pepe PE, Burch J, Feliciano D. Prospective MAST study in 911 patients. J. Trauma. 1989;29:1104-12
911 trauma patients > 15 years of age with systolic BP < 90 mmHg were randomized by alternate day assignment. All transported by HFD to Ben Taub hospital trauma center.
Experimental variable: MAST or no-MAST
Dependent variable: survival from trauma
Mattox KL, Bickell B, Pepe PE, Burch J, Feliciano D. Prospective MAST study in 911 patients. J. Trauma. 1989;29:1104-12
911 trauma patients > 15 years of age with systolic BP < 90 mmHg were randomized by alternate day assignment. All transported by HFD to Ben Taub hospital trauma center.
Experimental variable: MAST or no-MAST
Dependent variable: survival from trauma
ExampleExample
Independent variables: etiology, age, race, sex, location of injury, trauma scores, injury severity scores. Scores revealed the two groups to be statistically identically matched.
Findings: Mortality rate 31% in the MAST group and 25% in the non-MAST group.
Difference statistically significant.
Independent variables: etiology, age, race, sex, location of injury, trauma scores, injury severity scores. Scores revealed the two groups to be statistically identically matched.
Findings: Mortality rate 31% in the MAST group and 25% in the non-MAST group.
Difference statistically significant.
Double Blind StudyDouble Blind Study
A double blind study is one in which neither the patient nor the physician knows whether the patient is receiving the treatment of interest or the control treatment.
It is a type of RCT.
A double blind study is one in which neither the patient nor the physician knows whether the patient is receiving the treatment of interest or the control treatment.
It is a type of RCT.
Double Blind StudyDouble Blind Study
Bracken MB, Shephard MJ, Collins WF, et al. A randomized, controlled trial of methylprednisolone or naloxone in the treatment of acute spinal-cord injury. NEJM 1990;322(20):1405-11 Patients with spinal cord injury randomized by computer to receive methyprednisolone, naloxone, or placebo.Researchers and patients did not know which drug was which (prepared in pharmacy)
Bracken MB, Shephard MJ, Collins WF, et al. A randomized, controlled trial of methylprednisolone or naloxone in the treatment of acute spinal-cord injury. NEJM 1990;322(20):1405-11 Patients with spinal cord injury randomized by computer to receive methyprednisolone, naloxone, or placebo.Researchers and patients did not know which drug was which (prepared in pharmacy)
Quasi-Experimental StudiesQuasi-Experimental Studies
Quasi-experimental studies use intact groups of subjects rather than assigning subjects to groups at random.
Quasi-experimental studies use intact groups of subjects rather than assigning subjects to groups at random.
Quasi-Experimental StudiesQuasi-Experimental Studies
Non-randomized controlled trial:1 group receives intervention
1 group receives no-intervention (control)
Subjects assigned to groups by methods other than randomization.
Non-randomized controlled trial:1 group receives intervention
1 group receives no-intervention (control)
Subjects assigned to groups by methods other than randomization.
Cohort StudiesCohort Studies
A Cohort Study is a study in which patients who presently have a certain condition and/or receive a particular treatment are followed over time and compared with another group who are not affected by the condition under investigation.
A Cohort Study is a study in which patients who presently have a certain condition and/or receive a particular treatment are followed over time and compared with another group who are not affected by the condition under investigation.
Cohort StudiesCohort Studies
Framingham Heart Study:People in Framingham, MA have been followed for over 50 years.Conducted by National Heart, Lung, and Blood Institute (NHLBI) and Boston University.5,209 residents between 30 and 60 years of age initially enrolled.In 1971, 5,124 children (and their spouses) of the original cohort added.500 minority members have been added.
Framingham Heart Study:People in Framingham, MA have been followed for over 50 years.Conducted by National Heart, Lung, and Blood Institute (NHLBI) and Boston University.5,209 residents between 30 and 60 years of age initially enrolled.In 1971, 5,124 children (and their spouses) of the original cohort added.500 minority members have been added.
Cohort StudiesCohort Studies
Framingham Heart Study:Every two to four years, study participants are given extensive medical examinations including a medical history, blood tests and other sophisticated tests such as bone scans, eye exams and echocardiograms assessing multiple aspects of their current health status.
Framingham Heart Study:Every two to four years, study participants are given extensive medical examinations including a medical history, blood tests and other sophisticated tests such as bone scans, eye exams and echocardiograms assessing multiple aspects of their current health status.
Cohort StudiesCohort Studies
Framingham Heart Study (results):Risk factors for heart disease identified.
Better understanding of the effects of lifestyle.
More than 1,000 papers published significantly adding to the knowledge base of heart disease.
Study continues….
Framingham Heart Study (results):Risk factors for heart disease identified.
Better understanding of the effects of lifestyle.
More than 1,000 papers published significantly adding to the knowledge base of heart disease.
Study continues….
Case-Control StudiesCase-Control Studies
Case Control Studies: Case control studies are studies in which patients who already have a certain condition are compared with people who do not.
Case Control Studies: Case control studies are studies in which patients who already have a certain condition are compared with people who do not.
Case SeriesCase Series
A case series is a report on a series of patients with an outcome of interest. No control group is involved.
A case series is a report on a series of patients with an outcome of interest. No control group is involved.
Case ReportCase Report
A case report is a narrative report of an interesting case.A case report is a narrative report of an interesting case.
Historical StudiesHistorical Studies
Historical studies are systematized searches for the facts and then using the information to describe, analyze, and interpret the past.
Historical studies are systematized searches for the facts and then using the information to describe, analyze, and interpret the past.
Historical StudiesHistorical Studies
Bledsoe BE, Smith MG. Medical Helicopter Accidents in the United States: A 10-Year Review. J Trauma. 2004;56:1325-1329
Bledsoe BE, Smith MG. Medical Helicopter Accidents in the United States: A 10-Year Review. J Trauma. 2004;56:1325-1329
Historical StudiesHistorical Studies
Methods: Review of all air medical helicopter accidents in the US from 1997-2002 from the NTSB database.
Methods: Review of all air medical helicopter accidents in the US from 1997-2002 from the NTSB database.
Systematic ReviewsSystematic Reviews
A summary of the medical literature that uses explicit methods to systematically search, critically appraise, and synthesize the world literature on a specific issue.
A summary of the medical literature that uses explicit methods to systematically search, critically appraise, and synthesize the world literature on a specific issue.
Systematic ReviewsSystematic Reviews
Bledsoe BE. Critical Incident Stress Management (CISM): Benefit or Risk for Emergency Services? Prehosp Emerg Care. 2003;7:272-279
Bledsoe BE. Critical Incident Stress Management (CISM): Benefit or Risk for Emergency Services? Prehosp Emerg Care. 2003;7:272-279
Meta-AnalysisMeta-Analysis
A meta-analysis is a systematic review that uses quantitative methods to summarize the results.
A meta-analysis is a systematic review that uses quantitative methods to summarize the results.
Meta-AnalysisMeta-Analysis
Meta-analysis of RCTs represents the highest level of scientific evidence.
Allows for more objective appraisal of the evidence.
Reduces the possibility of false negative results.
Meta-analysis of RCTs represents the highest level of scientific evidence.
Allows for more objective appraisal of the evidence.
Reduces the possibility of false negative results.
Meta-AnalysisMeta-Analysis
van Emmerik AAP, Kamphuis JH, Hulsbosch AM, Emmelkamp PMG. Single-session debriefing after psychological trauma: a meta-analysis. Lancet. 2002;360:766-771
van Emmerik AAP, Kamphuis JH, Hulsbosch AM, Emmelkamp PMG. Single-session debriefing after psychological trauma: a meta-analysis. Lancet. 2002;360:766-771
Meta-Analysis (Observational Studies)Meta-Analysis (Observational Studies)
Less valid than RCTs.
Still valid in that it gives better information of the population as a whole.
Many EMS interventions cannot be tested with an RCT as it might be unethical to withhold care from the control group.
Less valid than RCTs.
Still valid in that it gives better information of the population as a whole.
Many EMS interventions cannot be tested with an RCT as it might be unethical to withhold care from the control group.
Meta-Analysis (Observational Studies)Meta-Analysis (Observational Studies)
Bledsoe BE, Wesley AK, Eckstein M, Dunn TM, O’Keefe MF. Helicopter transport of trauma patients: a meta-analysis. J Trauma. 2006;60:1257-1266
Observational meta-analysis examining validated trauma scoring systems in trauma patients transported from the scene to a trauma center.
Bledsoe BE, Wesley AK, Eckstein M, Dunn TM, O’Keefe MF. Helicopter transport of trauma patients: a meta-analysis. J Trauma. 2006;60:1257-1266
Observational meta-analysis examining validated trauma scoring systems in trauma patients transported from the scene to a trauma center.
Meta-Analysis (Observational Studies)Meta-Analysis (Observational Studies)
Conclusions:2 out of 3 trauma patients transported from the scene to a trauma center have minor injuries based on validated trauma scoring criteria.
1 out of 4 patients transported are discharged from the emergency department.
Conclusions:2 out of 3 trauma patients transported from the scene to a trauma center have minor injuries based on validated trauma scoring criteria.
1 out of 4 patients transported are discharged from the emergency department.
Animal StudiesAnimal Studies
Animal studies provide insight into biology.
For ethical reasons, some studies cannot be carried out on humans.
Computer modeling are replacing many animal studies.
Animal studies provide insight into biology.
For ethical reasons, some studies cannot be carried out on humans.
Computer modeling are replacing many animal studies.
Expert OpinionExpert Opinion
Expert opinions can take various forms:Systematic reviews
Narrative reviews
Pure opinion pieces
Expert opinions can take various forms:Systematic reviews
Narrative reviews
Pure opinion pieces
Rational ConjectureRational Conjecture
Lowest level of scientific validity.
But, overall very important.
“You don’t have to run a Chi-Square test on common sense”
Lowest level of scientific validity.
But, overall very important.
“You don’t have to run a Chi-Square test on common sense”
JournalsJournals
Not all journals are the same.
Impact rating:Cites
Articles
Readership
Not all journals are the same.
Impact rating:Cites
Articles
Readership
JournalsJournals
Journal Impact Rating (2005)
NEJM 44.016
Lancet 23.407
JAMA 23.332
Circulation 11.632
BMJ 9.052
Crit Care Med 5.077
Ann Emerg Med 2.782
Acad Emerg Med 1.789
J Trauma 1.722
StatisticsStatistics
Statistics are essential to any scientific analysis to eliminate bias and chance.
“Torture numbers and they’ll confess to anything.” (Gregg Easterbrook)
“Satan delights equally in statistics and in quoting scripture.... ” (H.G. Wells)
“The average human has one breast and one testicle.” (Des McHale)
Statistics are essential to any scientific analysis to eliminate bias and chance.
“Torture numbers and they’ll confess to anything.” (Gregg Easterbrook)
“Satan delights equally in statistics and in quoting scripture.... ” (H.G. Wells)
“The average human has one breast and one testicle.” (Des McHale)
SummarySummary
EMS must be driven by science.
Science is based upon quality research.
EMS providers of the future must be able to understand, and in certain cases, conduct valid research.
EMS must be driven by science.
Science is based upon quality research.
EMS providers of the future must be able to understand, and in certain cases, conduct valid research.
GoodGood
Gaucshe M, Lewis RJ, StrattonSJ, et al. Effect of Out-of-Hospital Pediatric Intubation on Survival and Neurologic Outcome: A Clinical Trial. JAMA. 2000;283:783-790
Gaucshe M, Lewis RJ, StrattonSJ, et al. Effect of Out-of-Hospital Pediatric Intubation on Survival and Neurologic Outcome: A Clinical Trial. JAMA. 2000;283:783-790
GoodGood
Methodology: RCT
Randomization: By calendar day.
Journal: JAMA
Participants: 830 consecutive patients 12 years and weight < 40 kg
Interventions: BVM only (odd days)
N= 410
BVM then ETI (even days)N= 420
Methodology: RCT
Randomization: By calendar day.
Journal: JAMA
Participants: 830 consecutive patients 12 years and weight < 40 kg
Interventions: BVM only (odd days)
N= 410
BVM then ETI (even days)N= 420
Outcome Measures:Survival to discharge
Neurologic status at discharge
Results:No improvement in survival or neurological outcome in either group.
Outcome Measures:Survival to discharge
Neurologic status at discharge
Results:No improvement in survival or neurological outcome in either group.
BadBad
Timmerman S, Cardoso LF, Ramires JAF, Halperin H. Improved Hemodynamic Performance with a Novel Chest Compression Device During Treatment of in-hospital Cardiac Arrest. Resusc. 2004;61:273-280
Timmerman S, Cardoso LF, Ramires JAF, Halperin H. Improved Hemodynamic Performance with a Novel Chest Compression Device During Treatment of in-hospital Cardiac Arrest. Resusc. 2004;61:273-280
BadBad
Methodology: Case series
Randomization: None.
Journal: Resuscitation
Participants: 31 terminally-ill, fresh dead Brazilians who failed 10 minutes of ACLS. Age 68 10 years
Interventions: Swan-Ganz catheters placed
Methodology: Case series
Randomization: None.
Journal: Resuscitation
Participants: 31 terminally-ill, fresh dead Brazilians who failed 10 minutes of ACLS. Age 68 10 years
Interventions: Swan-Ganz catheters placed
Interventions:90 seconds A-CPR (60 compressions / minute) followed by manual CPR (100 compressions / minute).
No ventilations
Subjects then intubated. Ventilated (12 / minute)
Epinephrine given
Interventions:90 seconds A-CPR (60 compressions / minute) followed by manual CPR (100 compressions / minute).
No ventilations
Subjects then intubated. Ventilated (12 / minute)
Epinephrine given
BadBad
Outcome Measures:CPP
MAP
Results:Usable blood pressures obtained in 16/31 patients (52%)
Rosary or wake interrupted (25%)
CPP:A-CPR: 20 12
M-CPR: 15 11
Outcome Measures:CPP
MAP
Results:Usable blood pressures obtained in 16/31 patients (52%)
Rosary or wake interrupted (25%)
CPP:A-CPR: 20 12
M-CPR: 15 11
PAP:A-CPR: 115 42
M-CPR: 153 28
Results:Increased CPP is related to increased chances of resuscitation. A-CPR improved CPP.
PAP:A-CPR: 115 42
M-CPR: 153 28
Results:Increased CPP is related to increased chances of resuscitation. A-CPR improved CPP.
UglyUgly
Everly GS, Flannery RB, Eyler V A. Critical Incident Stress Management (CISM): A Statistical Review of the Literature. Psych Quart. 2002;73:171-182
Everly GS, Flannery RB, Eyler V A. Critical Incident Stress Management (CISM): A Statistical Review of the Literature. Psych Quart. 2002;73:171-182
UglyUgly
Methodology: Meta-analysis?
Randomization: None.
Journal:
Participants: Meta-analysis of 8 CISM investigations.
Evaluation: Results pooled and Cohen’s d calculated
Methodology: Meta-analysis?
Randomization: None.
Journal:
Participants: Meta-analysis of 8 CISM investigations.
Evaluation: Results pooled and Cohen’s d calculated
Results: Significant effect size (Cohen’s d) of 3.11 showing a beneficial outcome from CISM.
Results: Significant effect size (Cohen’s d) of 3.11 showing a beneficial outcome from CISM.
d =v 2
1 +v 22 /2
mean1- mean2
You can’t obtain a Cohen’s d without a
control group!
UglyUglySTUDY CISM MEASURES PROBLEM
Busuttil (1995) Used CISM to treat PTSD in British Air Force personnel.
PTSD Scale CISM is not a treatment for PTSD
Flannery (1995) ASAP AssaultsQuestionnaire
No mention of CISM in paper
Flannery (1998) ASAP AssaultsQuestionnaire
No mention of CISM in paper
Flannery (1999) ASAP AssaultsQuestionnaire
No mention of CISM in paper
Richards (1999) Debriefing following bank robbery in NSW
IES CISM provided at 10 0r 24 hours
Mitchell (2000) CISM provided for volunteer firefighters with PTSD 3 years after a tornado
Diagnostic interviews CISM is not a treatment for PTSD
Flannery (2000) ASAP Assaults No mention of CISM in paper
Flannery (2000) ASAP Assaults CISM mentioned
GoodGood
Wang HE, Kupas DF, Hostler D, Cooney R, Yealey DM, Lave JR. Procedural Experience with out-of-hospital endotrachela intubation. Crit Care Med. 2005;33:
Wang HE, Kupas DF, Hostler D, Cooney R, Yealey DM, Lave JR. Procedural Experience with out-of-hospital endotrachela intubation. Crit Care Med. 2005;33:
GoodGood
Methodology: Analysis of administrative database
Randomization: None
Journal: Critical Care Medicine
Participants:Records: 1,544,791
11,484 ETIs by 5,245 rescuers.
Methodology: Analysis of administrative database
Randomization: None
Journal: Critical Care Medicine
Participants:Records: 1,544,791
11,484 ETIs by 5,245 rescuers.
Outcome Measures:Of 5,245 rescuers:
67% performed 2 ETIs.
39% performed 0 ETIs/
Mean ETIs = 1
Results:Out-of-hospital intubation is an uncommon event.
Outcome Measures:Of 5,245 rescuers:
67% performed 2 ETIs.
39% performed 0 ETIs/
Mean ETIs = 1
Results:Out-of-hospital intubation is an uncommon event.
BadBad
Amatangelo M, Thomas SH, Harrison T, Wedel SK. Analysis of Patients Discharged from Receiving Hospitals within 24 Hours of Air Medical Transport. Air Med J. 1997;16:44-14
Amatangelo M, Thomas SH, Harrison T, Wedel SK. Analysis of Patients Discharged from Receiving Hospitals within 24 Hours of Air Medical Transport. Air Med J. 1997;16:44-14
BadBad
Methodology: Retrospective review of flight records
Randomization: None
Journal: Air Medical Journal
Participants:945 flights
42 (4.4%) discharged less than 24 hours of transport.
Methodology: Retrospective review of flight records
Randomization: None
Journal: Air Medical Journal
Participants:945 flights
42 (4.4%) discharged less than 24 hours of transport.
Results: Patients met air transport criteria.
A priori
UglyUgly
Ohshige K, Kawakami C, Kubota K, Tochikubo O. A contingent valuation study of the appropriate user price for ambulance service. Acad Emerg Med. 2005;12:932-940
Ohshige K, Kawakami C, Kubota K, Tochikubo O. A contingent valuation study of the appropriate user price for ambulance service. Acad Emerg Med. 2005;12:932-940
UglyUgly
Methodology: Questionnaire survey
Randomization: None
Journal: Academic Emergency Medicine
Participants:
3,363 ambulance service customers sent survey.
2,029 (60.3%) returned survey.
Methodology: Questionnaire survey
Randomization: None
Journal: Academic Emergency Medicine
Participants:
3,363 ambulance service customers sent survey.
2,029 (60.3%) returned survey.
Results:Public inelastic (not concerned) about cost of ambulance service when case is an emergency.
Public fairly elastic about price in nonserious situations.
Thus, a ¥ 20,000 (US$ 190.00) user fee will reduce the number of nonemergency calls.
Results:Public inelastic (not concerned) about cost of ambulance service when case is an emergency.
Public fairly elastic about price in nonserious situations.
Thus, a ¥ 20,000 (US$ 190.00) user fee will reduce the number of nonemergency calls.
You don’t have to run a Chi
Square test on common sense.