Litigating Definitions of the “Relevant Scientific Community” Where There is no Science
Innocence Network Conference May 2, 2015
M. Chris FabricantDirector, Strategic Litigation
Innocence Project
Topics Why you should care about this talk.
How courts have traditionally defined relevant scientific communities
How those definitions have led to the admission of unvalidated and unreliable evidence, and to wrongful convictions
How the federal forensic agenda and hair microscopy audit can be leveraged to argue for expanded definitions of the relevant scientific community
Why should I care how a court defines a “relevant scientific community”?
Frye, Daubert and Postconviction Litigation of Unreliable/Discredited Scientific Evidence
Primary Areas of Litigation
Frye & Daubert litigation Decisions hinge on acceptance of the technique by the
“scientific community”
Post-conviction discredited science cases, e.g., hair microscopy, bite mark analysis, GSR, arson, etc., Decisions hinge on whether the technique remains
generally accepted in the defined scientific community.
Proffers of non-practitioner expert witness Persuading courts to admit testimony by experts outside
of the technique at issue, e.g. Mnookin/Cole
A Central Frye/Daubert Issue Re “Scientific” Evidence, as Defined by Simon Cole
How will courts define the relevant scientific
community and measure general acceptance
where “a non-scientific expert offer[s] testimony
about what purports to be scientific evidence
without an obvious scientific community to which a
fact-finder can refer for an assessment of the
validity of this testimony”?
The Frye Standard As interpreted…
Novel expert testimony based on scientific principles or procedures is admissible only where the "'techniques, when properly performed, generate results accepted as reliable within the scientific community generally.'" Parker v. Mobil Oil Corp., 7 N.Y.3d 434, 446 (2006) (emphasis added)
As pronounced…
In Frye, which involved a challenge to the admissibility of a lie detector test, the court precluded the evidence because it had not yet “gained such standing and scientific recognition among physiological and psychological authorities as would justify the courts in admitting expert testimony deduced from the discovery, development, and experiments thus far made.”
Daubert and…Kumho Tire Litigation? (1) whether the theory or technique in question can be and
has been tested; (2) whether it has been subjected to peer review and publication; (3) its known or potential error rate; (4) the existence and maintenance of standards controlling its operation; and (5) whether it has attracted widespread acceptance within a relevant scientific community.
“[I]t would prove difficult, if not impossible, for judges to administer evidentiary rules under which a gatekeeping obligation depended upon a distinction between ‘scientific’ knowledge and ‘technical’ or ‘other specialized’ knowledge. There is no clear line that divides the one from the others.”
Kumho Tire Co. Ltd. v. Carmichael, 526 U.S. 137, 148 (1999)
Discredited Science in Postconviction Texas
Tex. Code Crim. Proc. art. 11.073 previously unavailable scientific evidence discredits scientific evidence relied on by the State at trial.
California §1473 Penal Code amended to address expert opinion which are
”repudiated by the expert who originally provided the opinion”; “False evidence” now includes opinions “undermined by later scientific research or technological advances.”
Federal Court “Slow and painful has been man’s progress from magic to law” -
Han Tak Lee State evidentiary rulings are guided by due process, not Daubert.
Everywhere else Change in scientific understanding is “newly discovered” evidence
establishing the technique is no longer generally accepted in the relevant scientific community.
Proffer of Relevant Experts/Academics Lacking “Real World” Experience Courts frequently exclude experts who have
not actually conducted the type of forensic test at issue:
Prof. Jennifer Mnookin: See Earnest v. Commonwealth, 61 Va.App 223 (2012)
Prof. Simon Cole: State v. Armstrong, 920 So.2d 769 (2006)
NOTE: Expansive definitions of the relevant scientific community should be leveraged to argue these experts have relevant expertise which the court and/or the jury should consider in weighing admissibility or the weight of evidence issues.
The Community of Discredited Scientists Comparative Bullet Lead Analysis/Neutron
Activation Analysis Microscopic Hair Comparison Bite Mark Comparison Evidence Voiceprint Gunshot Residue Historical Cellphone Tower Tracking
Discredited Science = Discredited Precedent
Cases of first impression concerning the admissibility of many of these discredited techniques typically defined the relevant community as comprised of the experts themselves. (But see Frye!)
“Those who ‘generally accepted’ the discipline . . . constituted the relevant community whose general acceptance was required for admissibility.”
The Shifted Paradigm (VA Journal of Criminal Law, forthcoming)(collecting cases)
A “Scientific” Community of Dentists: The History of Bite Mark Jurisprudence
Every case of first impression defined the relevant scientific community as the dentists engaged in the novel technique
Bite mark evidence has contributed to a24 wrongful convictions/indictments
Arguments for Expansive Definitions Post-DNA era shatters myths surrounding
admissible, yet unvalidated and unreliable
The composition of the 2009 NAS Committee
The FBI/DOJ hair comparison case audit
Composition of NCFS
Composition of various SACs, OSACs and the FSSB
0%
10%
20%
30%
40%
50%
60%
70%
80%72%
47%
27%
15%
Contributing Factors in the DNA Exonerations
(first 325)
Unreliable Techniques Subject to Scrutiny by Objective Scientists from Relevant Fields CBLA review by the NAS. The reviewing panel
included: Practitioners Criminalist (Peter Deforest – early critic of hair evidence) Chemists Statistics & Probability experts (K. Kafadar; C. Speigleman)
Voice spectrography review by the NAS.: Police and academic practitioners Chaired by an acoustical engineer. Engineers Physiologists Linguistics experts Statistics & Probability experts
The NAS’s Influence on the Courts
CBLA: See, e.g., Clemons v. State, 896 A.2d 1059, 1070 (Md. 2006) (citing an NAS report questioning the scientific basis for CBLA evidence in determining that evidence should be excluded under Frye); accord Ragland v. Commonwealth, 191 S.W.3d 569, 580 (Ky. 2006); State v. Behn, 868 A.2d 329, 331 (N.J. Super. Ct. App. Div. 2005).
Polygraph: In re Jordan R., 140 Cal. Rptr. 3d 222, 238 (Ct. App. 2012) (affirming a lower court's exclusion of polygraph evidence based in part on a 2003 NAS report criticizing the scientific underpinnings of such polygraph testing).
DNA: The NAS generated two reports on DNA evidence, one in 1992 and another in 1996.
Nat’l Academy of Sciences, Strengthening Forensic Science in the United States: A Path Forward (2009)
The 2009 NAS Committee Included:ChemistsBiologistsComputer ScientistsPhysicistsStatistics & ProbabilitiesAnd practitioners
Validation: The Scientific Community’s Role in Forensic Science
“One particular task of science is the validation of new methods to determine their reliability under different conditions and their limitations. Such studies begin with a clear hypothesis (e.g., “new method X can reliably associate biological evidence with its source”).” NAS Rpt. p. 113
The Scientific Community’s Conclusion re Most Venerable Forensic Techniques
“With the exception of nuclear DNA analysis...no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.”
The FBI Adopts the NAS’s - i.e. the Scientific Community’s - Conclusions re Hair Microscopy
“An examiner report or testimony that applies probabilities to a particular inclusion of someone as a source of a hair of unknown origin cannot be scientifically supported. Such testimony exceeds the limits of science and is therefore inappropriate.” The FBI limits of science statement
“No scientifically accepted statistics exist about the frequency with which particular characteristics of hair are distributed. There appear to be no uniform standards on the number of features on which hairs must agree before an examiner may declare a ‘match.’” NAS Report
A Post-NAS Report Argument for a Narrow Definition of the “Scientific Community”
“The make-up of the relevant scientific community is and
should be those who have the knowledge, training and
experience in bitemark analysis and who have actually
done real world cases. We enter a looking-glass world
when the defense urges that the Court ignore the opinions
of working men and women who make up the ranks of
board-certified forensic odontologists, who respond to
emergency rooms and morgues, who retrieve, preserve,
analyze and compare evidence, who make the reports
and who stand by their reasoned opinions under oath.”
As to the NAS Report….The defense would instead have this Court rely on the opinions of statisticians, law professors and other academics who do not and could not do the work in question.
When Dr. Kafadar and her NAS committee created the NAS report, they wrote a summary assessment of forensic odontology. In it they said that "the majority of forensic odontologists are satisfied that bite marks can demonstrate sufficient detail for positive identification ...”**
**The entire quote: “Although the majority of forensic odontologists are satisfied that bite marks can demonstrate sufficient detail [f]or positive identification, no scientific studies support this assessment, and no large population studies have been conducted.”
What is (Not) the “Relevant” Scientific Community? Not limited to the practitioner’s themselves
Cell Tower: Given that multiple factors can affect the signal strength of a tower and that Special Agent Raschke's chosen methodology has received no scrutiny outside the law enforcement community, the court concludes that the government has not demonstrated that testimony related to the granulization theory is reliable. U.S. v. Evans, : 892 F.Supp.2d 949 (N.D.Ill 2012)(historical cell tower tracking)
CBLA: “At the time of appellant's trial, the scientific community consisted of two people in the FBI. Ward v. State, 108 A.3d 507 (MD App 2015)
Additional Support for Broadening the Community Beyond Practitioners.
People v. Wesley, 83 N.Y.2d 417, 438 (1994) (Kaye, J., concurring) (If the field is too narrowly defined, the judgment of the scientific community will devolve into the opinion of a few experts. The field must still include scientists who would be expected to be familiar with the particular use of the evidence at issue, however, whether through actual or theoretical research.”)
Reed v. State, 391 A.2d 364, 377 (Md. 1978)(“voiceprint” includes linguists, physiologists, psychologists, and statisticians)
State ex rel. Collins v. Superior Court, 644 P.2d 1266, 1285 (Ariz.1982) (“This requirement is not satisfied with testimony from a single expert or group of experts who personally believe the challenged procedure is accepted or is reliable.”)
Contreras v. State, 718 P.2d 129, 135 (Alaska 1986)(”We define the relevant scientific community as the academic, scientific, and medical or health-care professions which have studied and/or utilized hypnosis for clinical, therapeutic, research and investigative applications.”)
“Scientific Community” Cannot be Defined by Interested Expert Witnesses. Wesley, 83 N.Y.2d at 438-39 (Kaye, J., concurring)(A court must be wary of
the "pitfalls of self-validation by a small group"; before being generally accepted, methods must have been subjected to "scrutiny of fellow scientists, unimpeded by commercial concerns.“)
State v. Thompkins, 891 So.2d 1151, 1152 (Fla. Dist. Ct. 2005)(“[t]he only testimony was from two people who earn a living by giving polygraph tests.”)
People v. Coy, 669 N.W.2d 831, 838 (Mich. Ct. App. 2003) (per curium)(“When demonstrating that there is general scientific recognition of novel scientific techniques or principles, it is necessary to present the testimony of disinterested and impartial experts whose livelihood is not intimately connected with the technique at issue.”)
People v. Young, 391 N.W.2d 270, 276 n.24 (Mich. 1986)(“Scientific community approval is absent where those who have developed and whose reputation and livelihood depends on use of the new technique alone certify, in effect self-certify, the validity of the technique.”)
“THREE LEGGED STOOL”
National CommissionOn Forensic
Science(NCFS)
Organization Of Scientific Area Groups
(OSAC)
National Institute ofStandards and
Technology(NIST)
Roles explained in Memorandum of Understanding (MOU) between DOJ and NIST.
Nat’l Commission on Forensic Sciences “The objectives and scope of activities of the
Commission are to provide recommendations and advice to the Department of Justice (DOJ) concerning national methods and strategies for: strengthening the validity and reliability of the forensic sciences (including medico-legal death investigation): enhancing quality assurance and quality control in forensic science laboratories and units: identifying and recommending scientific guidance and protocols for evidence seizure, testing, analysis, and reporting by forensic science laboratories and units; and identifying and assessing other needs of the forensic science communities to strengthen their …”
Consistently Relevant Fields of Science Statistics & Probabilities Experts
Essential due to their expertise in how to design, analyze, and interpret performance data, i.e.: Validation research Proficiency testing Measurement standards Critique of inherently probabilistic conclusions - See
Hair Audit
Cognitive Scientists How the human mind processes “domain irrelevant”
information How cognitive bias/observer effects influences
conclusions
Does the ”technique generate results accepted as reliable”, i.e. validated through the scientific method?
Scientific Validation The validity of any forensic test means that the test
does what it purports to do. What are the capabilities, limits, measures of
uncertainty of the test?
Demonstrated Reliability Given the same sample/evidence, how consistent
are the results of the forensic test? If the forensic test is repeated under different
conditions – quality of the evidence, experience of the examiner – are the conclusions the same?
How is General Acceptance established?
“[B]y citing to peer-reviewed literature in his/her particular field. This is accomplished by demonstrating that independent studies have been performed utilizing this particular theory and that the results have been duplicated, and that the studies were done on a statistically significant number of subjects”
Guest v. NYC, 2013 WL 6989226 (N.Y. Sup.)