construct validity of bitemark assessments using...
TRANSCRIPT
Overview of presentation
• Background • Scientific approach • Methods • Results • Impact and suggestions
Background
The decision tree is a means of formalizing the approach to bitemark analysis by taking the assessor through a series of stages and decisions that aim to ensure that the decisions made are consistent with the level of forensic evidence available.
Background – Schematic of tree
Patterned Injury
Is it a human BM ? Not a human BMSuggestive of a
human BM
Analysis – No comparison
Human Bitemark No analysis, no comparison
What are the characteristics ?
Next Step
Background
This study examined Step 1 – the evaluation of the injury, is the injury a bitemark and if so, what are the bitemark’s characteristics? Today presenting data on the assessment of the injury as a bitemark only
Scientific approach Several methods being applied to BM research: • Mechanistic approach • Decision making approach
In the absence of truth we are using construct validity – through reliability testing - if its not reliable its not valid.
Methods 250 cases submitted by DABFO – included an orientation shot and a close up with scale Selected 100 cases to represent a wide spread of anatomical location, presentation, evidence quality Presented to DABFO on an online system with anonymity of decisions Asked if there was sufficient evidence to render any opinion, and if so, what is it?
Methods Data collected Demographics reported Kappa used to measure agreement Descriptive statistics to assess the spread of decisions and understand the reasons for disagreement
Results 38 Diplomates completed the whole study, 44 completed partially. Represents a total of 3924 decisions on bitemark cases Range of experience measured in three ways:
How many cases in past 5 years – 18.58 How many years have you been active – 19.87 How many times have you testified in the past five years? – 2.05
Results – decision spread Look at the spread of decisions for individual cases.
-‐2
3
8
13
18
23
28
78
68
81
86
65
70
52
85
99
15
100 69
18
35 4 43
98
60
94
56
73
44
66
36
83
22
55
62
89 3 41
77
30
51
76
10
11
21
27
47
48
92
96
72 8 74 1 50
19
25
This is a human bite mark This is sugges;ve of a bite mark This is not a bite mark
Results – decision spread Look at the spread of decisions for individual cases.
0
5
10
15
20
25
30
35
40
34 49 71 90 95 29 97 2 40 5 75 24 39 53 84 91 23 31 37 87 12 17 32 67 93 13 38 46 6 79 82 28 58 59 64 9 14 20 42 45 57 88 16 54 61 80 26 33 63 7
This is a human bite mark This is sugges;ve of a bite mark This is not a bite mark
Impact and suggestions All research has strengths and weaknesses • Good number of decisions • A lot of work – not all diplomates completed • Some argued that not realistic approach
The study suggests level of reliability of injury assessment for bitemarks is not currently satisfactory from the population of assessors studied
The impact of three choices from the decision model has decreased reliability– removal of “suggestive” combined with greater detail on the identification of bitemarks within the decision model should be considered. The use of a simpler, dichotomous decision, should lead to increased reliability – although the decision direction of the “suggestives” is unknown.
Impact and suggestions We need to undertake further examination of those cases where there was higher levels of agreement to determine how the decision tree can capture these elements to improve reliability, both for bites and non-bites. The first step of BM analysis is determining if the presented injury is a bitemark – the current data suggest that agreement levels observed require significant improvement and means of achieving this have been proposed. A further assessment following the introduction of these changes will be required.