Identifying the probable cause

2

Define the Case

List Candidate Causes

Evaluate Data from the Case

Evaluate Data from Elsewhere

Identify Probable Cause

Detect or Suspect Biological Impairment

As Necessary: Acquire Data,

and Iterate Process

Identify and Apportion Sources

Management Action: Eliminate or Control Sources, Monitor Results

Biological Condition Restored or Protected

Decision-maker and

Stakeholder Involvement

Stressor Identification

3

How we identify probable causes

• Eliminate when you can• Diagnose when you can• Otherwise, analyze strength of evidence

Apply a scoring system to the available evidence under each type of evidence

HOW?

4

The scoring system

R refutes

D diagnoses

+++ convincingly supports (or weakens - - -)

++ strongly supports (or weakens - -)

+ somewhat supports (or weakens - )

0 neither supports nor weakens

NE no evidence

5

What you are looking for spatial/temporal co-occurrence

+The effect occurs where or when the candidate cause occurs, OR the effect does not occur where or when the candidate cause does not occur.

0 It is uncertain whether the candidate cause and the effect co-occur.

- - - The effect does not occur where or when the candidate cause occurs, OR the effect occurs where or when the candidate cause does not occur.

R

The effect does not occur where and when the candidate cause occurs, OR the effect occurs where or when the candidate cause does not occur, and the evidence is indisputable.

6

Why score this way

+This finding somewhat supports the case for the candidate cause, but is not strongly supportive because the association could be coincidental.

0 This finding neither supports nor weakens the case for the candidate cause, because the evidence is ambiguous.

- - - This finding convincingly weakens the case for the candidate cause, because causes must co-occur with their effects.

R This finding refutes the case for the candidate cause, because causes must co-occur with their effects.

7

no R Because laboratory tests do not exactly replicate toxic effects in the field.

Scoring of laboratory tests of site media

Scoring of stressor-response from other field studies

no RBecause there may be substantial and consistent differences in organisms or conditions between the case and elsewhere.

8

Willimantic case studyMetals NH3 Flow Silt Low

DOTemp Food Episodic

Mix

Types of Evidence that Use Data from the Case

Spatial/Temporal Co-Occurrence + - + - - - + +

Evidence of Biological Mechanism + + + - + + - +

Causal Pathway - + - - + - +

Stressor-Response from the Field + - - + +

Manipulation of Exposure + + +

Verified Predictions + + +

Types of Evidence that Use Data from Elsewhere

Stressor-Response from Other Field - - +

Stressor-Response from Laboratory + + - - +

Example: strength of evidence analysis

9

Weighing the evidence for each candidate cause

• Evaluate the quantity & quality of evidence• Evaluate consistency, patterns, strength, &

credibility• Summarize the compelling evidence

10

Evaluate quantity & quality of evidence

• Quality & quantity of data influence scores

• Now evaluate overall quality of evidence

• Lots of consistent evidence reduces quality concerns for any 1 line of evidence

• Poor quality data may be discounted

• Consider study designs, methods, relevance, variability, & other QA issues

11

Evaluate consistency, patterns, strength, & credibility

• Prepare summary table of scores

• Do not add up scores!

• Evaluate consistency of evidence

• Look for compelling evidence

• If evidence is inconsistent, consider mechanistic explanations

– e.g., lab data not consistent with field conditions due to differing bioavailability

12

Scoring consistency & credibility

Consistency of Evidence

All available types of evidence support the case for the candidate cause. + + +

All available types of evidence weaken the case for the candidate cause. - - -

All available types of evidence support the case for the candidate cause, but few types are available. +

All available types of evidence weaken the case for the candidate cause, but few types are available. -

The evidence is ambiguous or inadequate. 0

Some available types of evidence support and some weaken the case for the candidate cause. -

Explanation of the Evidence

There is a credible explanation for any negative inconsistencies or ambiguities in an otherwise positive body of evidence that could make the body of evidence consistently supporting.

+ +

There is no explanation for the inconsistencies or ambiguities in the evidence. 0

There is a credible explanation for any positive inconsistencies or ambiguities in an otherwise negative body of evidence that could make the body of evidence consistently weakening.

-

13

Willimantic case studyMetals NH3 Flow Silt Low

DOTemp Food Episodic

Mix

Types of Evidence that Use Data from the Case

Spatial/Temporal Co-Occurrence + - + - - - + +

Evidence of Biological Mechanism + + + - + + - +

Causal Pathway - + - - + - +

Stressor-Response from the Field + - - + +

Manipulation of Co-occurrence + + +

Verified Predictions + + +

Types of Evidence that Use Data from Elsewhere

Stressor-Response from Other Field - - +

Stressor-Response from Laboratory + + - - +

Evaluating Multiple Types of Evidence

Consistency of Evidence - - - - - + - + + +

14

Summarize compelling evidence

• Make an overall evaluation of strength of evidence for each candidate cause

– what evidence compels belief that candidate cause induced effect?

– what evidence strongly casts doubt?

• Consider the principle characteristics of causal relationships

– these are what you’re trying to show– they summarize the 15 types of evidence

15

Physical Interaction

Causal agents change an affected agent by physical interaction.

Directionality The cause precedes its effectSufficiency Intensity or frequency of contact with the

cause is adequate to produce observed effect

Sequential Dependence

All effects result from a prior sequence of cause-effect events

Coherence of Characteristics

Specific causal relationships are consistent with scientific theory

Characteristics of Causal Relationships

16

There is no magic formula…

All candidate causes must be compared to determine:

– if there is more than 1 probable cause– the level of confidence in the results

17

Comparing evidence among causes: best-case scenario

You have compelling evidence for 1 candidate cause; others are weak or refuted...

TYPE OF EVIDENCE

CANDIDATE CAUSE

1 2 3

A ++ - R

B + --

C + -

Consistency + -

…celebrate, then remediate for Candidate Cause 1

18

Comparing evidence among causes: more (likely) scenarios

You have uneven evidence across candidate causes...

TYPE OF EVIDENCE

CANDIDATE CAUSE

1 2 3

A ++ –

B + –

C + +

Consistency + –

• Strong evidence for one candidate cause may be sufficient• Consider if weakness is due to lack of data

19

You have unsatisfying evidence across all candidate causes…

TYPE OF EVIDENCE

CANDIDATE CAUSE

1 2 3

A + –

B – –

C +

Consistency – – –

• Reconsider the impairment• Consider additional candidate

causes• Consider gathering more data

• Consider episodic events• Apply professional judgment

to identify most likely cause

20

You have evidence suggesting multiple causes…

TYPE OF EVIDENCE

CANDIDATE CAUSE

1 2 3

A ++ + +

B + + ++

C ++ ++ +

Consistency + + +

• Consider disaggregating indices or metrics• Combine causes if they share causal pathways, modes of

action, sources and routes of exposure, or if they interact• Remediate dominant cause• Design remediation to address multiple causes

21

You have insufficient data…

TYPE OF EVIDENCE

CANDIDATE CAUSE

1 2 3

A NE NE –

B + NE NE

C NE NE NE

Consistency 0 0 0

• Gather data if possible• Consider other bases for remediation (e.g., BMPs,

chemical criteria) and monitor biological responses• Use professional judgment as last resort

22

How do I communicate results?

• Make your logic clear

• Present the critical evidence

• Reveal uncertainties

• Fit communication to your audience

• For technical reviewers, include text & tables

• For decision makers, may be helpful to use annotated conceptual model

23

↑ other toxics

Example: using models for communication

↓ dissolved oxygen

↑ temperature

↓ EPT richness ↓ trout abundance

POTWindustrial facility dams subdivisiondairy farm

↑ NH3↑ Zn

DO higher at impaired site vs. reference

after rerouting industrial discharge had decreased concentrations of Zn & other toxics, increased EPT taxa richness

NH3 higher at impaired site vs. reference, but not at levels sufficient to cause impairment

industrial facility

24

What comes after causal analysis?

• If confidence in results is low…– plan studies to obtain critical evidence– experimental studies most likely to be convincing

• If confidence in results is high…– identify sources– take action– monitor results