ENFSI 24 th September 2014The value of trace evidence.

Dr. David Lucy

d.lucy@lancaster.ac.uk

Lancaster University

ENFSI - 2014 – p.1/38

Epistemology

Epistemology - theories of knowledge, or , how we canknow things.

• What distinguishes a scientific knowledge from otherforms of knowledge is: how we come to knowsomething is equally important to what it is we know.

• The sciences appeal to observation and to logic asguides to truth, or at least to identify falsehood.

Both physical sciences and law share a reliance onepistemology.

ENFSI - 2014 – p.2/38

Epistemology

In science:

• We do not allow “arguments by authority”,

• those are truths the truth value of which depends uponthe person saying it.

The same is not really true in law:

• Eyewitness testimony is a very valuable source of legalevidence.

• Not really authority in the same way.

ENFSI - 2014 – p.3/38

Epistemology

In science:

• Most of our scientific knowledge is from others,

• a very small amount of our scientific knowledge is atfirst hand.

• Have to have trust those others are communicatingactual observations.

In law:

• Hearsay (secondhand) evidence is forbidden,

• Information illegally obtained is not allowed asevidence.

ENFSI - 2014 – p.4/38

Epistemology

In a physical science:

• We can simply repeat our experiments,

• in principal we can repeat our observations infinitely

Irrespective of what a single set of observations mightindicate, a large number of repeats will eventually lead to aconsensus, and provisionally this consensus will representthe truth with the current information.

In science a thoroughgoing epistemology is not reallyneeded to practice science.

ENFSI - 2014 – p.5/38

Forensic science

Legal sciences are “historical” sciences:

• Not usually possible to any level of replication,

• each case is unique, an ontological singularity,

• not repeated in principal, not just difficult, impossible.

This means that legal epistemology must emphasisemethod.

In the absence of replication to avoid falsehood the onlydevices we have are our scientific principles of knowledgeacquisition.

ENFSI - 2014 – p.6/38

Comparison problem

The “comparison problem” is the archetypal problem offorensic science:

• Where some trace is left at the scene of a crime.

• Some similar trace has been found to be associatedwith a suspect.

To what extent do the observations from the suspects itemconvince one that the crimescene item and suspect itemare one and the same.

Addresses source level propositions. Applies to someextent to all members of WG.

ENFSI - 2014 – p.7/38

Glass

ENFSI - 2014 – p.8/38

Glass

ENFSI - 2014 – p.9/38

Glass

With glass the observables can be:

• refractive index,

• major, minor and trace element measurements,

• isotopic measurements - being done for bullet leads(Knut & KRIPOS).

All the above are termed continuous measurements:

• do not fit in categories.

Looking at two sub-samples of observations of refractiveindex from a single glass jar.

ENFSI - 2014 – p.10/38

Glass RI

Glass refractive indicies for a jar (Greg’s data):

1.5176 1.5178 1.5180 1.5182

5

10

15

20

RI

prob

abili

ty d

ensi

ty(×

1000

)

jar 63 − subsample 1jar 63 − subsample 2

ENFSI - 2014 – p.11/38

Proximity

The closeness of observation:

• All the observations from sub-sample one fit withinsub-sample two,

• sub-sample one and sub-sample two can be said tomatch.

The p-value is 0.6757 which is the probability of seeing thisdifference in the means of the two sub-samples were theytruly from a population with a single mean.

Is this a good measure of whether the sub-samples arefrom the same glass item?

ENFSI - 2014 – p.12/38

Proximity

An easily conceived illustration:

• A friend collects coloured marbles - stores them inbags.• each bag has marbles of only one colour.• there may be more than one bag containing

marbles of each colour.

• A marble has dropped out of a bag - the marble is red -you select a bag and sample a marble - that marble isred.

To what extent does the observation that both marbles arered support the notion that the marble came from that

particular bag? ENFSI - 2014 – p.13/38

Proximity

Complete match of "red"

Does this mean that the marble came from bag three?

ENFSI - 2014 – p.14/38

Proximity

Probability marble came from bag three is one.

ENFSI - 2014 – p.15/38

Proximity

?

Probability marble came from bag three is one third (13).

ENFSI - 2014 – p.16/38

Proximity

Similarity of observation on its own:

• Gives no idea as to identity.

Dissimilarity of observation:

• Can be used to reject in cases where observation isunambiguous.

• Does not apply in any logical manner whereobservations are continuous.

Need knowledge of population to make any legitimateprobabilistic inference about identity of source.

ENFSI - 2014 – p.17/38

Continuous observations

Have just seen how proximity of observation gives littleevidence for identity of source:

• this simplified the case by considering discreeteobservations,

• that is: marble was red, blue, turquoise etc.

• As the marble from the bag was red, and the recoveredmarble red, then we had an absolute “match”.

To what extent is the notion of a match in this sense a realexpectation with truly continuous observations?

ENFSI - 2014 – p.18/38

Glass RI

Glass refractive indicies for two jars (Greg’s data):

1.5178 1.5180 1.5182

5

10

15

RI

prob

abili

ty d

ensi

ty(×

1000

)

jar 63jar 71

ENFSI - 2014 – p.19/38

RI variation

Why the variation in RI observation?

• Is Greg a poor observer? no,

• is the GRIM process imprecise? a bit,

• do the observation conditions change? they do.

If the conditions better, and a more precise GRIMapparatus purchased, would the observations of RI from asingle piece of glass be all the same?

Unlikely!

ENFSI - 2014 – p.20/38

Glass RI

Can we logically exclude values occurring with lowprobability?

1.5178 1.5180 1.5182

5

10

15

RI

prob

abili

ty d

ensi

ty(×

1000

)

jar 63jar 71

p=1 × 10−8

ENFSI - 2014 – p.21/38

Uncertainty in observation

sample 1sample 3

sample 2

Finite samples will always give different observations:stochastic process.

ENFSI - 2014 – p.22/38

Uncertainty in observation

Kinetic theory implies variation in repeated sampling fromthe same item:

• variation is a consequence of the material,

• has a stochastic origin.

• regardless of the precision of apparatus,

The consequences are that for a continuously varyingquantity an exact match for two samples taken from thesame item is very unlikely.

ENFSI - 2014 – p.23/38

Summary so far

Thus far two salient facts have emerged for comparisonproblems:

1. proximity alone can give little idea of identity,

2. for continuous variables, an exact match from twosub-samples of the same item is very unlikely.

Both the above are from deep principle - not just inpractice.

What do we do about this? Can we live with theuncertainties?

ENFSI - 2014 – p.24/38

Likelihood

Repeated sampling of jar 63 leads to a “distribution”.

1.51780 1.51790 1.51800 1.51810

5

10

15

RI

prob

abili

ty d

ensi

ty(×

1000

) 0.04

ENFSI - 2014 – p.25/38

Likelihood

Probabilities need care:

• there is a 4% probability of observing a RI between1.517899 and 1.517905.

• Given that all the observations are taken from jar 63,

• or can say “conditioned on” the observation being fromjar 63.

It is all too easy to say that there is a 4% probability that aRI between 1.517899 and 1.517905 is from jar 63 - This isutterly wrong.

ENFSI - 2014 – p.26/38

Likelihood

If you wanted to know the probability that you had Jar 63were you to observe a RI between 1.517899 and1.517905, then:

• you would have to observe a large sample (population)of RIs between 1.517899 and 1.517905,

• then count how many of those were from Jar 63.

• Which as an empirical experiment you cannot do.

When thinking about probabilities we have to be veryprecise about what events in the world the probabilityrefers to.

ENFSI - 2014 – p.27/38

Likelihood

Sampling from some of the other glass objects:

1.5176 1.5180 1.5184

2

4

6

8

10

12

RI

prob

abili

ty d

ensi

ty(×

1000

)

0.02

ENFSI - 2014 – p.28/38

Likelihood

Both Jar 63 and other items:

Jar 63

1.5176 1.5180 1.5184

2

4

6

8

10

12

RI

prob

abili

ty d

ensi

ty(×

1000

)

other glass items

ENFSI - 2014 – p.29/38

Likelihood ratio

For a point just take ratio of heights:

Jar 63

1.5176 1.5180 1.5184

2

4

6

8

10

12

RI

prob

abili

ty d

ensi

ty(×

1000

)

other glass items

2624.68

12211.5

ENFSI - 2014 – p.30/38

Likelihood ratios

This is why statisticians use likelihood ratios:

• they’re simple,

• they focus on the probabilities of the observations,

• they do not try to calculate probabilities for which youcannot make any direct observations.

By considering a likelihood ratio we automatically take intoaccount the likelihood of the measurements given somecomparative material and the populational items.

We also evaluate evidence - not propositions.

ENFSI - 2014 – p.31/38

Multivariate

Most continuous observations are multivariate:

35 36 37 38 39

0.75

0.80

0.85

0.90

0.95

1.00

1.05

208Pb/ 204Pb

207 P

b/20

6 Pb

4.8

other bullets

208Pb/ 204Pb35 36 37 38 39

67.65

bullet PMC

ENFSI - 2014 – p.32/38

State of art in UK

Methods available for source level comparisons:

• never actually used in court to my knowledge (Greg!),

• however, my knowledge is limited.

• Exclusions despite not being logical are pretty safe,

For those which are not excluded courts seem happy with“cannot rule out” types of evaluations.

Which are completely true, but, as we have seen, arewithout much meaning.

ENFSI - 2014 – p.33/38

Where to go next?

Software:

1. “comparison” package available for R:• comparison is really for specialists only,• does provide S4 class objects - these form the core

programs

2. ENFSI funded NFI project to make it all more availableto practitioners.

ENFSI - 2014 – p.34/38

Where to go next?

Methological:

1. databases - absolutely needed• never going to answer the questions of comparing

items without;• those who argue that it is too expensive cannot

even begin to answer the question!

2. Curse of dimensionality - more than about fivedimensions/elements difficult.

Integration with other sources of evidence to elevate toactivity level. Greg has already started.

ENFSI - 2014 – p.35/38

Final comments

The latest ENFSI guidelines are:

• only guidelines - unlikely to compel scientists to adoptall proposals,

• part of the immediate future for ENFSI members.

I hope this talk has been a useful guide to the ideas behindthe proposals and the rationale for their adoption.

Acknowledgements: thanks to Greg Zadora (IFR, Cracow) and Knut-Endre Sjåstad(KRIPOS, Oslo) for their help in the preparation of this talk; and Delia Kingsbury for theinvitation.

ENFSI - 2014 – p.36/38

The bags example

Pr(red|bag3) = 1Pr(red|otherbag) = 0

LR = 1

0= ∞

The evidence of a red marble from bag three, and red “lost”marble, is infinitely more likely were the source of themarble bag three than any other bag

ENFSI - 2014 – p.37/38

The bags example

Pr(red|bag3) = 1

Pr(red|otherbag) = 2

4= 0.5

LR = 1

0.5= 2

?

The evidence of a red marble from bag three, and the red“lost” marble, is twice as likely were the source of themarble bag three than any other bag

ENFSI - 2014 – p.38/38