expert evidence the need for vigilance?. it is not shocking that courts can overly rely on...

Expert Evidence

The need for Vigilance?

• It is not shocking that courts can overly rely on apparently reliable scientific evidence.

• Studies show it has a tremendous effect on juries.

• It seems independent and unbiased, but is it?

Author’s Conclusion

• A reliable method of protecting the system from junk science is needed.

• Legal standards alone are insufficient without understanding the scientific methodology, language, and interpretive basis from which the evidence is derived.

Role of Counsel

• To question scientific validity, convey scientific understanding.

Expertise

• An expert must have qualifications that enable them to comment based on specialized training, knowledge, and/or expertise that the trier of fact lacks.

• The area in which this expertise is based is defined as a “branch of study in a [discipline] concerned with a connected body of demonstrated truths or with observed facts systematically classified and more or less connected together by a common hypothesis operating under general laws. The branch should include trustworthy methods for the discovery of new truths within its own domain.

Exploring Reliability

• Whether trying to have expert evidence admitted or excluded, reliability is a crucial consideration. Scientific validity and reliability are critical elements that underscore legal reliability. If the evidence tendered cannot meet standards of scientific competence, it cannot purport to have sufficient merit to be considered reliable for the purposes of adjudication.

J.(L.L.) – Novel Science

(1) Whether the theory or technique has been tested, and whether an attempt was made at falsification;

(2) Whether the theory or technique has been subject to publication and peer review;

(3) the known or estimated rate of error;

(4) General acceptance within the discipline.

• In J.(L.L.), these criteria were used to exclude expert opinions based on the penile plethysmograph.

• There was no distinct psychological profile, nor could the test indicate whether a person was a member of the distinctive class of deviants.

J.(L.L.)

• Point: the scientific method itself exposed the shortcomings of the procedure at issue.

• The success rate for identifying a sexual deviant was slightly les than 50%.

• With the potential for 50% false negatives, the test was of questionable probative value (no better than a flip of the coin) as a means to identify sexual deviants.

J.(L.L.)

• Error rate: both a means to judge the scientific reliability of a conclusion, but also legal reliability.

Forensic Science Evidence

• Not produced to generate data for study or to falsify a hypothesis, but to produce evidence.

• Some traditional and now well-established means of gathering forensic evidence were simply never submitted to modern scientific scrutiny and are only recently being challenged for their lack of scientific foundation.

Forensic Science: Problems Identified

(a) Lack of understanding of scientific methodology;

(b) Inability of scientist to explain findings in clear and accurate terms;

(c) Failure to conduct thorough examinations;

Cont’d

(d) Inability to design meaningful experiments in order to reach meaningful conclusions about the evidence;

(e) A willingness to tailor testimony to meet the needs of counsel.

• Conclusion: challenging other than novel forensic science for a lack of scientific methodology is made more difficult by the historical acceptance from which forensic investigation benefits.

• There is a higher level of scrutiny for non-forensic science.

Examiner Bias

• Forensic Investigation especially susceptible to examiner bias or confirmatory bias – ie. the effect of a goal on the person performing a study seeking confirmation, not falsification.

• The goal becomes the creation of evidence, often against a specific target sample.

• The pressure to produce results and confirm a suspect also leads to false positives.

• E.g. the Washington attorney linked to a terrorist bombing in Madrid by three senior fingerprint examiners – stress of high profile case, and influence of other identifications.

Other examples

• A number of convictions against parents for “shaking” their children to death, when their deaths were a result of natural causes.

• Evidence included the opinion of the “statistical improbability” that more than one death in a family could have resulted from natural causes.

Concern

• The theories we accept and rely on now have often proven wrong at a later time, yet we have relied on them for determination of legal matters.

• Query: have these theories and practices been put to the same scientific scrutiny as non-forensic science would do?

Hair and Fiber Comparison

• Used to convict Guy Paul Morin.

• Morin inquiry: “valueless.”

• At best, circumstantial. Its limitations not generally understood.

Hair comparison

• Human-based comparison, depending on skill, training and subjective assessment of individual examiner

• An exclusionary, not inclusionary test – it can exclude donors, but is of limited use in telling who the donor was.

• Limited probative value: it merely permits the trier of fact to infer the accused is one of a limitless class of persons who cannot be excluded

Fiber Comparison

• Similarities may occur because of direct contact between a victim and suspect, or from other secondary contact between each of them individually and source fibers. The comparison test cannot support any conclusion about the source of the similar-seeming fibers.

Fiber Comparison

• Secondary vs. Primary transfer: does it depend on number of fibers? No.

• Kaufman: not good science.

• In 8/28 cases of wrongful conviction studied, DNA evidence exonerated individuals whose convictions were based on hair and/or fiber evidence.

Room for Improvement?

• The legal profession must become more scientifically literate.

• Courts historically looked to scientific evidence as objective and comparatively certain, and thus it was unnecessary to ask: “how do you know?” But the question must be asked.

E.g. DNA testing

• Not immune from error rates and improper test procedures.

• Understanding the demands of scientific testing and the meaning of error rates remain essential.

E.g. Fingerprint Analysis

• Done with the human eye with the aid of a microscope. The criteria used to conclude the existence of a match depends on the subjective perspective of the particular examiner and not on scientific procedure.

Point

• Understanding the scientific method will enable counsel to not only see but to explain to the jury through cross-examination the potential fallibility of the proposed method of testing and to examine the error rates.

Science 101: for Lawyers

• Approach expert opinion evidence by posing certain fundamental questions on the admissibility inquiry.

• Good science is premised on careful testing that provides proof of a hypothesis through a rigorous effort to disprove it. Once the hypothesis cannot be disproved, then the hypothesis or theory attains a reliability.

Null Hypothesis

Composed in order to be refuted, to further support the alternative or proposed hypothesis.

A vital step in statistical significance testing and probability statements.

Scientific Process

• Formulation of hypothesis, testing hypothesis in order to disprove it, reliability from the failure to disprove it.

• Basic requirement: the process must be subject to replication.

• The hypothesis and challenge should be subject to peer review (allows other to test same).

Counsel

• Use the steps in the scientific inquiry to evaluate expert evidence and identify elements that are most vulnerable.

• Use the following five (5) inquiries.

One

• Has foundational theory or technique been exposed to the scientific method? Or does it rely on untested assumptions?

• e,.g. the comparison of bullet metallurgic composition assumed each vat of lead used for bullet-making resulted in distinct and unique compositions of metal, and that in the manufacturing of bullets, there was no mixing of different vats on different days.

• Those assumptions were completely false.

• This evidence, used to tie ammunition in possession of accused, with bullet fragments at the scene, resulted in many convictions.

• Example: “These two chemically indistinguishable bullets probably originated from the same box of ammunition.”

• Tobin and Randich, formerly of the FBI, found that contrary to what they thought, the composition of castings derived from the same pot of molten metal sometimes deviated from one another while lead composition from different pots was sometimes a match. In addition, bullets from one source pot could easily become mixed up with bullets from a different original source. E.g. discovering boxes of ammunition with 14 distinct metallurgic groups meant that the previous “matches” were of no probative value.

Two

• Have the results of the test been reproduced? Or can they be? All scientific results must be capable of reproduction.

• E.g. symptoms of child sexual abuse includes behaviors arising from family breakdown – ie. admission of expert these cannot be controlled for.

• In physical sciences, external factors can include contamination, inadequate preservation of sample, experimenter bias, lack of proper inference being drawn.

Three

• Rate of error of test, and put that into probative vs. prejudicial value language. There is an error rate for all scientific testing. An unstated or miscalculated error rate allows an expert to present conclusions under a false glow of accuracy. Tests with an unknown error rate are a harbinger of disaster.

Four

• What lab produced the result? What is its accreditation? Were proper procedures followed? What level of training does staff have? What controls are in place to guard against human error etc.? Is any of the testing done double-blind? Has the lab produced questioned results in the past?

Five

• Has the underlying theory or methodology used to come to the conclusion been subject to scrutiny by scientific peers?

• Seek out scientific journal articles that form part of the peer review process.

• The Court is not in a position to be a peer review examiner.

Linguistic Precision

• Expert science contributes to wrongful convictions not only because of questions to do with the science, but also because of a general lack of ability on the part of the scientific community to convey information with comprehensible precision.

• Statements made are ripe for misinterpretation and over-reliance.

e.g.

• “Consistent with” properly means only that the two observations or phenomenon are not inconsistent.

• Query: does the TOF appreciate this distinction?

Practice Point

• Counsel has to insure that the conclusions are expressed by the expert are understood in such a way that their limitations are evident to the TOF. A tentative conclusion may be very dangerous if the TOF misunderstands its significance.

“Match”

• Misleading, often overstates their connection

• E.g. in hair comparison a “Type 1 match” is only that it may or may not have come from the donor.

Problem: Lack of Remedy?

• What is the sanction for the junk scientist?

Science: Not Certain

• All science has room for human error, lab error, investigator bias or simply bad science