Don’t Blame the Forensic Scientist!

Roger G. KopplInstitute for Forensic Science Administration, FDU

Robert Kurzban

University of Pennsylvania

Lawrence KobilinskyJohn Jay College of Criminal Justice

DisclaimerResearch for this talk was supported in part

by the National Science Foundation. Any opinions, findings, and conclusions or

recommendations expressed in this paper are those of the authors and do not necessarily

reflect the views of the National Science Foundation.

Problem: Error creates a blame game

Error is a perceived problemand a real problem

• Persons wrongly accused• Labs performing poorly • Proficiency tests• Dror & Charlton studies• MD v Rose

When errors are discovered• Blame is a common response• Similar blaming language is used in dissimilar

cases:– Fred Zane “got away with 12 years of sloppy work

and false testimony” (“Reasonable Doubt: Can Crime Labs Be Trusted?” CNN 1/13/05)

– Jacqueline Blake neglected her negative controls in DNA tests. “This went on for more than two years before she was finally caught.” (“Encore Presentation: Reasonable Doubt” CNN 5/7/06)

• Kelly M. Pyrek: Forensic Science Under Siege, – Academic Press, 2007

In the blame game we replace one bad model of the forensic scientist

with another.

The first bad model imagines forensic scientists to be wonderful

creatures who are never make mistakes . . .

The second bad model imagines forensic scientists to be bumbling incompetents who should not be allowed anywhere

near criminal evidence. . .

Blame follows glory

Our model . . .

In other words• Forensic scientists are:

–real, fallible human beings–Whom we should not blame in an

emotional manner–But study in a scientific manner

• Leading to a very different take on error

The Blame Game is Bad

• “It is time to end this destructive, fatalistic blame game and replace it with a new agenda of constructivism.”– Pyrek, Forensic Science Under Siege, p. 526

• Pyrek rightly cites “To Err is Human: Building a Safer Health System”– Institute of Medicine of the National

Academies, 1999

“To Err is Human” p. 49

• The common initial reaction when an error occurs is to find and blame someone. However, even apparently single events or errors are due most often to the convergence of multiple contributing factors. Blaming an individual does not change these factors and the same error is likely to recur. Preventing errors and improving safety for patients require a systems approach in order to modify the conditions that contribute to errors.

James Reason:

“We cannot change the human condition, but we can change the conditions under which humans work.”

Error

• Is not a personal problem– You’re bad

• Requiring a personal solution– Be good

• It is a structural problem– Monopoly epistemics

• Requiring a structural solution– Democratic epistemics– Structural redundancy

Monopoly vs. Democratic Epistemics

• A crime lab has a monopoly on the analysis of evidence sent to it

• That’s “monopoly epistemics”• Sometimes evidence should be sent to

more than one crime lab• That’s “democratic epistemics”

Monopoly Epistemics

message set Sender Receiver judgment

Match Crime Lab Jury The latent was leftNo match by the suspectinconclusive

Democratic Epistemics

.

message set senders receiver judgment

Match Crime Labs Jury The latent was leftNo match by the suspectinconclusive

Monopoly Epistemics vs.

Democratic Epistemics

• Other studies address cost– Errors are costly: e.g. $100,00 – Forensic tests are cheap: e.g. $50 – Structural redundancy would reduce taxpayer

costs -- if it works to reduce errors• Our experiments test if it works

Basics of Experimental Design

• Sender(s) evaluate evidence and give a report to a Receiver

• Receiver guesses the truth given the report(s)

• Sender(s) shown one of three shapes:

• Receiver guesses which shape was shown

Basics of Experimental Design

• Receiver paid $5.00 for correct guesses, $2.00 for incorrect guesses

• Senders are paid $3.00 for inducing correct guesses

• Senders are also shown a “supplementary shape” worth $1.00 or $5.00 depending on the round.

What the Sender Sees

018

The supplementary shape is: The value of the supplementary shape is: $1.00 The correct shape is:

On the following page, place a check mark on the shape you wish to report to DM2. Use the marker provided.

What the Sender Fills Out

018

Report Form One of the DM1 has reported the shape with the check mark on it.

Monopoly Treatment

• Subjects paired into Senders and Receivers• Senders informed of “correct shape,”

“supplementary shape,” the value of supplementary shape

• A $1.00 for the supplementary shape gives Senders an incentive to issue an accurate report

• A $5.00 value gives Senders an incentive to issue an inaccurate report -- if the bias does not agree with truth

Democratic Treatments

• Each Receiver is matched with 2, 3, 4, or 5 Senders

• Otherwise like monopoly treatment • Major question:

– Do Receiver guesses improve under democratic epistemics?

Results: Receiver Errors

• Receiver error rates across all conditions

– 1 Sender: 43%– 2 Senders: 35%– 3 Senders: 18%– 4 Senders: 18%– 5 Senders: 21%

• Receiver error rates if Senders have high bias– 1 Sender: 51%– 2 Senders: 55%– 3 Senders: 33%– 4 Senders: 36%– 5 Senders: 43%

So What?

• Structural redundancy works• Three is the magic number• Other research shows it will be cost

effective

Closing Remarks

• Forensic Science is a bargain for the criminal justice system, and we need more of it.