SQO 4/7/05

INCORPORATING MULTIPLE LINES OF EVIDENCE INTO

SEDIMENT QUALITY OBJECTIVES

Stephen B. Weisberg

Southern California Coastal WaterResearch Project

SQO 4/7/05

BACKGROUND

• For many years, scientist have advocated a triad approach for evaluating sediment quality– Individual lines of evidence each have potential limitations– Most applications have been site-specific and based on best

professional judgment

• There are many challenges in translating scientific concept into a state-wide regulatory framework– Standardizing interpretation– Ph.D. biologists vs. B.S. engineers

SQO 4/7/05

LARGEST CONCERNS YOU EXPRESSED

• Reconsider using a linear, numerical scoring system– Implies a linearity that may not exist– Assures equal weighting among different lines or evidence

• Enhance interaction with stakeholders– Integration framework is a combination of science and policy

• Place greater emphasis on application guidance

RELATIONSHIP BETWEEN SCIENCE AND POLICY

• There are a spectrum of decisions– Selecting indicators and thresholds from individual lines of evidence– Joining multiple lines of evidence to make a station assessment– Joining multiple stations to make a water body (on site) assessment

• Science is about linearizing complex information

• Policy is about establishing thresholds along those gradients– Policy becomes predominant at the higher levels of information

SQO 4/7/05

Species Species AbundanceAbundancePseudopolydora paucibranchiata 612 Diplocirrus sp SD1 4Exogone lourei 465 Ampithoe sp 3Fabricinuda limnicola 240 Asthenothaerus diegensis 3Musculista senhousia 170 Euchone limnicola 3Caprella californica 113 Heteroserolis carinata 3Scoletoma sp 106 Lyonsia californica 2Scoletoma sp C 99 Neotrypaea californiensis 2Solen rostriformis 92 Sabellidae 2Amphideutopus oculatus 51 Acteocina inculta 1Podocerus fulanus 50 Aplousobranchia 1Amphipholis squamata 41 Ceriantharia 1Prionospio heterobranchia 40 Eteone brigitteae 1Paradexamine sp SD1 27 Glycera americana 1Mayerella acanthopoda 23 Halcampidae 1Spiophanes duplex 20 Heterophoxus ellisi 1Edwardsiidae 17 Macoma sp 1Pista percyi 17 Malmgreniella sp 1Ericthonius brasiliensis 15 Monocorophium acherusicum 1Leitoscoloplos pugettensis 14 Monocorophium insidiosum 1Oligochaeta 12 Nassarius tiarula 1Leptosynapta sp 11 Odontosyllis phosphorea 1Mediomastus sp 11 Paranemertes californica 1Megalomma pigmentum 10 Protohyale frequens 1Phoronida 8 Pyromaia tuberculata 1Leptochelia dubia 7 Scleroplax granulata 1Rudilemboides stenopropodus 6 Zoobotryon pellucida 1Theora lubrica 5 Imogine exiguus 1Anoplodactylus erectus 4 Planoceridae 1Stylochidae 1

RELATIONSHIP BETWEEN SCIENCE AND POLICY

• There are a spectrum of decisions– Selecting indicators and thresholds from individual lines of evidence– Joining multiple lines of evidence to make a station assessment– Joining multiple stations to make a water body (on site) assessment

• Science is about linearizing complex information

• Policy is about establishing thresholds along those gradients– Policy becomes predominant at the higher levels of information

SQO 4/7/05

LARGEST CONCERNS YOU EXPRESSED

• Reconsider using a linear, numerical scoring system– Implies a linearity that may not exist– Assures equal weighting among different lines or evidence

• Enhance interaction with stakeholders– Integration framework is a combination of science and policy

• Place greater emphasis on application guidance

APPLICATION GUIDANCE

• Moving from site-based objectives to water body assessments

• Selecting implementation options where there are no chemical-specific thresholds

• Consider a sequential of phased sampling approach where effort is proportional to the nature of the problem

SQO 4/7/05

LARGEST CONCERNS YOU EXPRESSED

• Reconsider using a linear, numerical scoring system– Implies a linearity that may not exist– Assures equal weighting among different lines or evidence

• Enhance interaction with stakeholders– Integration framework is a combination of science and policy

• Place greater emphasis on application guidance

SQO 4/7/05

MLOE WORKPLAN

• Task 1: Define the lines of evidence that will be used for each beneficial use– Select the indicators for those lines of evidence

• Task 2: Develop a scoring system for each LOE– When a single indicator is available– When multiple indicators are available

• Task 3: Integrate scoring across multiple LOEs to develop a station assessment

• Task 4: Develop a strategy for incomplete data

• Task 5: Develop application guidance

BENEFICIAL USE PROTECTION CATEGORIES

• Aquatic life (Infaunal) effects

• Human health effects

• Fish and wildlife effects

• MLOE will be used within each, but independent assessments will be conducted for each beneficial use

SQO 4/7/05

MLOE WORKPLAN

• Task 1: Define the lines of evidence that will be used for each beneficial use– Select the indicators for those lines of evidence

• Task 2: Develop a scoring system for each LOE– When a single indicator is available– When multiple indicators are available

• Task 3: Integrate scoring across multiple LOEs to develop a station assessment

• Task 4: Develop a strategy for incomplete data

• Task 5: Develop application guidance

SQO 4/7/05

CLASSIC TRIAD APPROACH

• Score each LOE as a binary decision

• This yields 8 narratively-interpretable integrated endpoints

• Relies heavily on best professional judgement– Assessment tools are too crude for binary decisions– Eight endpoints don’t capture uncertainty within individual LOE

TYPICAL TRIAD INTERPRETATION

Chem Tox Benthos Interpretation

+ + + Impact highly likely: Contaminant-induced degradation in field evident

+ + - Impact likely: Toxic contaminants probably stressing sediment-dwelling organisms

- + + Impact likely: Unmeasured chemicals contributing to toxicity

+ - + Impact likely: Toxicity test not sensitive enough

+ - - Impact unlikely: Contaminants unavailable to organisms in the field

- + - Impact unlikely: Unmeasured factors contributing to toxicity

- - + Impact unlikely: Effects on benthos not due to sediment contamination

- - - Impact highly unlikely: Contaminant-induced degradation not evident

SQO 4/7/05

CLASSIC TRIAD APPROACH

• Score each LOE as a binary decision

• This yields 8 narratively-interpretable integrated endpoints

• Relies heavily on best professional judgement– Assessment tools are too crude for binary decisions– Eight endpoints don’t capture uncertainty within individual LOE

SQO 4/7/05

ALTERNATIVE APPROACH

• Create multiple categories for each line of evidence– Recognize uncertainty associated with a single threshold

• Leads to many more than 8 combinations– A headache, but provides more information

• Allows for assessment of endpoints that describe magnitude of effects

• Also allows for endpoints that describe confidence in the result– Confidence based on level of effect or agreement among individual

lines of evidence

SQO 4/7/05

AGREEMENT ABOUT CATEGORIES

• Individual lines of evidence

• MLOE site assessment

LIKELY ASSESSMENT CATEGORIES FOR INDIVIDUAL LINES OF EVIDENCE

• Reference condition

• Slight deviation from reference– Probably defined by measurement error

• Moderate effect

• Severe effect

SQO 4/7/05

AGREEMENT ABOUT CATEGORIES

• Individual lines of evidence

• MLOE: Site assessment

SQO 4/7/05

POSSIBLE SITE ASSESSMENT CATEGORIES

• Unimpacted

• Likely unimpacted

• Possibly impacted

• Likely impacted

• Clearly impacted

• Inconclusive?

• Reference

• Unimpaired

• Unimpacted

• Unaffected

• Protected

• Affected– Individual line of evidence

• Impacted– Site level– Merging multiple lines of evidence

• Impaired– Water body level– Merging multiple sites

SQO 4/7/05

POSSIBLE SITE ASSESSMENT CATEGORIES

• Unimpacted

• Likely unimpacted

• Possibly impacted

• Likely impacted

• Clearly impacted

• Inconclusive?

SQO 4/7/05

CHEMISTRY: Reference

Reference Minordeviation

Moderateeffect

Severeeffect

Reference Unimpacted UnimpactedLikely

UnimpactedInconclusive

Minordeviation

UnimpactedLikely

UnimpactedPossibly Impacted

Possibly Impacted

Moderateeffect

LikelyUnimpacted

Possibly Impacted

Possibly Impacted

LikelyImpacted

Severeeffect

Inconclusive Possibly Impacted

LikelyImpacted

LikelyImpacted

Benthos

Toxicity

SQO 4/7/05

CHEMISTRY: Minor Deviation

Reference Minordeviation

Moderateeffect

Severeeffect

Reference UnimpactedLikely

UnimpactedPossibly Impacted

Possibly Impacted

MinorDeviation

Likely Unimpacted

Possibly Impacted

Possibly Impacted

Likely Impacted

ModerateEffect

Possibly Impacted

Possibly Impacted

Likely Impacted

Likely Impacted

SevereEffect

Possibly Impacted

Likely Impacted

Likely Impacted

Likely Impacted

Benthos

Toxicity

SQO 4/7/05

CHEMISTRY: Moderate Effect

Reference Minordeviation

Moderateeffect

Severeeffect

Reference Likely Unimpacted

Possibly Impacted

Likely Impacted

Likely Impacted

MinorDeviation

Possibly Impacted

Possibly Impacted

Likely Impacted

Likely Impacted

ModerateEffect

Likely Impacted

Likely Impacted

Clearly Impacted

Clearly Impacted

SevereEffect

Likely Impacted

Likely Impacted

Clearly Impacted

Clearly Impacted

Benthos

Toxicity

SQO 4/7/05

CHEMISTRY: Severe Effect

Reference Minordeviation

Moderateeffect

Severeeffect

Reference Inconclusive Possibly Impacted

Likely Impacted

Likely Impacted

MinorDeviation

Possibly Impacted

Likely Impacted

Likely Impacted

Likely Impacted

ModerateEffect

Likely Impacted

Likely Impacted

Clearly Impacted

Clearly Impacted

SevereEffect

Likely Impacted

Likely Impacted

Clearly Impacted

Clearly Impacted

Benthos

Toxicity

SQO 4/7/05

NUMERICAL SYSTEM

• Can be simpler than look-up charts– Also can provide more gradation for prioritization or trends analysis

• Assumes a linearity

• Assumes an equal weighting

• Assumes the stakeholders community counts it

SQO 4/7/05

MLOE WORKPLAN

• Task 1: Define the lines of evidence that will be used for each beneficial use– Select the indicators for those lines of evidence

• Task 2: Develop a scoring system for each LOE– When a single indicator is available– When multiple indicators are available

• Task 3: Integrate scoring across multiple LOEs to develop a station assessment

• Task 4: Develop a strategy for incomplete data

• Task 5: Develop application guidance

SQO 4/7/05

APPLICATION GUIDANCE

• Inherently a policy issue

• Stakeholder advisory committee has been tasked to do this

• They have formed subcommittees to develop guidance for three applications– 303d listings– Dredging decisions– NPDES permitting

SQO 4/7/05

TECHNICAL SUPPORT

• Uncertainty analysis

• Sampling design guidance

• Phased sampling design

• Guidance for identifying contaminants of concern

SQO 4/7/05

UNCERTAINTY ANALYSIS

• Assessing uncertainty at the station level– Measurement variability– Assessment tool error

• Defining background level of “impact”– No system is likely to have completely unimpacted sites– Identify systems (or subsystems) that are least influenced– Use project data base to create a frequency plot for those systems

Station Score0 1 2 3 4 5 6 7 8 9

% o

f Site

s

0

5

10

15

20

25

30

35

SQO 4/7/05

TECHNICAL SUPPORT

• Uncertainty analysis

• Sampling design guidance

• Phased sampling design

• Guidance for identifying contaminants of concern

SQO 4/7/05

IDENTIFYING CONTAMINANTS OF CONCERN

• Chemical specific thresholds– Don’t want to rely on chemistry as a sole means for impairment

decision, but chemical thresholds can be useful guidance for implementation actions

• Spatial gradient analyses– Probably better in concept than practice

• Sediment TIES– This is a key area for scientific advance in support of management

SQO 4/7/05

MLOE WORKPLAN

• Task 1: Define the lines of evidence that will be used for each beneficial use– Select the indicators for those lines of evidence

• Task 2: Develop a scoring system for each LOE– When a single indicator is available– When multiple indicators are available

• Task 3: Integrate scoring across multiple LOEs to develop a station assessment

• Task 4: Develop a strategy for incomplete data

• Task 5: Develop application guidance

SQO 4/7/05

WHICH INDICATORS?

• Infaunal effects– Sediment chemistry– Sediment toxicity– Benthic infaunal assemblage

• Human health– Sediment chemistry– Fish/bivalve tissue chemistry

• Fish/wildlife– Sediment chemistry– Tissue exposure– Biological effects

SQO 4/7/05

MULTIPLE INDICATORS WITHINA LINE OF EVIDENCE

• Multiple toxicity tests available for the site– Chronic and acute tests

• Multiple ways to interpret the same data– Equilibrium partitioning vs. empirical thresholds

• Several alternatives for integrating such data

SQO 4/7/05

POSSIBLE SCORING APPROACHES

• Average score

• Worst score– They each measure different things

• Prioritizing among tests– Most sensitive test– Least sensitive test– “Best” test

• Highest quality data

• Some combination of frequency and severity

SQO 4/7/05

PROPOSED SCORING WHEN MULTIPLE INDICATORS ARE MEASURED

Severe effect Severe effect for one indicator and concordance among indicators

Moderate effect Severe effect for one indicator or concordance among indicators

Minor deviation Measured effect in at least one test

Reference Concordance among all tests that there is no effect

SQO 4/7/05

WHAT IF ONLY TWO LINES OF EVIDENCE ARE AVAILABLE?

• Looking for a combination of concordance and magnitude

• Unimpacted: No effect from either indicator

• Likely unimpacted: Small effect with no indicator, but no effect for the other

• Inconslusive: Large effect with one indicator but no effect for the other

• Likely impacted: Some effect for both indicators

• Clearly impacted: High effect for both indicators