the wild, wild, wet! setac expert advisory panel performance evaluation and data interpretation
TRANSCRIPT
THE WILD, WILD, WET!
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
THE PERFECT WORLD
FOCUS ON DATA ANALYSIS
• STEP 1: STEP 1: GRAPH THE DATAGRAPH THE DATA
• STEP 2: STEP 2: Analyze the data by EPA flowchartsAnalyze the data by EPA flowcharts
• STEP 3: STEP 3: DO THE RESULTS MAKE SENSE?DO THE RESULTS MAKE SENSE?
SOFTWARE PROGRAMS
• Many software packages/programs are Many software packages/programs are availableavailable
• DO NOT assume they follow the EPA DO NOT assume they follow the EPA recommended analysisrecommended analysis
• DO verify the software by running DO verify the software by running example datasets from the methods example datasets from the methods manualsmanuals
STATISTICAL AND BIOLOGICAL SIGNIFICANCE
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
TOXIC VS. NON-TOXIC
• How are data from a WET test used to How are data from a WET test used to make a decision of toxicity?make a decision of toxicity?
– Two paths:Two paths:• Decision based on the observed result• Decision based on standard effect
WHO DECIDES WHICH PATH?
• The Permit WritersThe Permit Writers– BOTH approaches are supported by the BOTH approaches are supported by the
TSD and the methods manualTSD and the methods manual
OBSERVED RESULT
• Data from the test are used to Data from the test are used to determine if toxicity is present by determine if toxicity is present by hypothesis testinghypothesis testing
– HHOO: Effluent is not toxic: Effluent is not toxic
– HHaa: Effluent is toxic: Effluent is toxic
STANDARD EFFECT
• A A preselectedpreselected level of effect is level of effect is considered toxicconsidered toxic– Acute test:Acute test: 50 % effect50 % effect– Chronic test:Chronic test: 25 % effect25 % effect
THERE ARE INHERENT STRENGTHS AND
WEAKNESSES TO BOTH APPROACHES
COMPONENTS WHICH IMPACT THE NOEC
WHAT BIOLOGICAL CONCLUSIONS CAN BE
MADE FROM THE STATISTICAL ANALYSIS OF A SINGLE TOXICITY TEST?
The biological impact was The biological impact was significant in the beakersignificant in the beaker
THE LESS THAN PERFECT WORLD
INTRA- AND INTER-TEST VARIABILITY
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
TYPES OF VARIABILITY
• Intra-test : among and between Intra-test : among and between concentrationsconcentrations
• Inter-test: within one lab, same methodInter-test: within one lab, same method
• Inter-lab: between labs, same methodInter-lab: between labs, same method
• Method specific: within limits of methodMethod specific: within limits of method
INTRA-TEST VARIABILITY
Group N Mean s.d. CVcontrol 4 0.975 0.050 0.051
2 4 0.975 0.050 0.051
3 4 0.975 0.050 0.051
4 4 0.950 0.058 0.061
5* 4 0.675 0.150 0.222
6* 4 0.275 0.222 0.806
MSE = 0.033% %MSD = 13.9 %
INTRA-TEST VARIABILITY AND ENDPT. UNCERTAINTYEC Concentration Upper
95% CLLower
95% CL1 330 110 515
10 569 286 769
50 1107 841 1390
90 2156 1662 3731
99 3712 2509 9639
POINT ESTIMATE INTER-TEST VARIABILITY
5
6
7
8
9
10
11
12
13
1 3 5 7 9 11 13 15 17 19
Tests
LC
50 (
mg/
l SD
S)
LC50
95% UCI
95% LCI
HYPOTHESIS TEST INTER-TEST VARIABILITY
0
50
100
150
200
250
1 2 3 4 5 6 7 8 9
Tests
NO
EC
(p
pb
Cu)
NOEC
SOURCES OF INTRA-TEST VARIABILITY
• Genetic variabilityGenetic variability
• Organism handling and feedingOrganism handling and feeding
• Toxicity among and between treatmentsToxicity among and between treatments
• Non-homogeneous sample sourceNon-homogeneous sample source
SOURCES OF INTRA-TEST VARIABILITY
• Abiotic conditionsAbiotic conditions
• Dilution schemeDilution scheme
• Number of organisms/treatmentNumber of organisms/treatment
• Dilution water pathogensDilution water pathogens
SOURCES OF INTER-TEST VARIABILITY
• Intra-test sourcesIntra-test sources
• Analyst experience and practiceAnalyst experience and practice
• Organism age and healthOrganism age and health
• AcclimationAcclimation
• Dilution waterDilution water
SOURCES OF INTER-TEST VARIABILITY
• Sample qualitySample quality
• Test chamber characteristicsTest chamber characteristics
SOURCES OF INTER-TEST VARIABILITY
• Replicate volumeReplicate volume
• ProceduresProcedures
VARIABILITY AND POINT ESTIMATE UNCERTAINTY
Test #1 Test #2
Mean CV (%) 9.9 33.8
IC25 (%) 27.2 26.0
MSE 34.5 290.6
95% CI 25.7-28.5 17.2-31.3
HIGH VARIABILITY - LOW STATISTICAL POWER
Group n Mean(ug/ind)
s.d. CV%
Control 4 632 552 87.4
2 4 727 674 92.7
3 4 1080 408 37.7
4 4 564 493 87.5
5 4 748 235 31.4
% MSD = 131 %
LOW VARIABILITY - HIGH STATISTICAL POWER
Group n Meansurvival
s.d. CV
Control 8 1.000 0.000 0.000
2 8 1.000 0.000 0.000
3 8 1.000 0.000 0.000
4 8 1.000 0.000 0.000
5 8 1.000 0.000 0.000
6* 8 0.950 0.093 0.148
% MSD = 1.0 %
ACTIONS TO REDUCE VARIABILITY
• Increase number of reps/treatmentIncrease number of reps/treatment
• QA programQA program
• Establish and follow strict proceduresEstablish and follow strict procedures
• Maximize analyst skillMaximize analyst skill
• Contract lab selectionContract lab selection
• Additional QA/QC criteriaAdditional QA/QC criteria
EXAMPLES OF ADDITIONAL QC TEST CRITERIA
• Region IX: upper MSD limitsRegion IX: upper MSD limits
• Washington: upper MSD limits, Washington: upper MSD limits, change in alphachange in alpha
• N. Carolina: limit control CVs, N. Carolina: limit control CVs, C. dubia “PSC”C. dubia “PSC”
• Region VI: limit control CV, Region VI: limit control CV, increase # replicates,increase # replicates,
biological significancebiological significance
SUSPICIOUS DATA AND OUTLIER DETECTION
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
CONCERNS
• Outliers make interpretation of WET Outliers make interpretation of WET data difficult bydata difficult by– Increasing the variability in test responsesIncreasing the variability in test responses– Biasing mean responsesBiasing mean responses
IDENTIFYING OUTLIERS
• Graph raw data, Graph raw data, means and means and residualsresiduals
Raw Data and Means
Copper Concentration (ppb)0 100 200 300 400
Pro
po
rtio
n A
live
0.0
0.2
0.4
0.6
0.8
1.0
Residuals
Copper Concentration (ppb)0 100 200 300 400
Re
sid
ua
l (p
red
icte
d -
ob
se
rve
d)
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
IDENTIFYING OUTLIERS• Formal statistical test - Chauvenet’s CriterionFormal statistical test - Chauvenet’s Criterion
– Using the previous mysid data, the critical values are:Using the previous mysid data, the critical values are:• Mean = .80, Std. Dev. = 0.302, n = 8
– Chauvenet’s Criterion Value = n/2 = 4Chauvenet’s Criterion Value = n/2 = 4
– Z-score = Z-score = 2.054 (two-tailed probability of 4 %)2.054 (two-tailed probability of 4 %)– The calculations are:The calculations are:
• Equation 1) (Z-score)(Std. Dev.) = (2.054)(0.302) = 0.620• Mean Equation 1 = 0.80 0.620 = 1.42 - 0.18• Outlier Range is >1.42 or <0.18
– A value of 0.2 is not an outlier.A value of 0.2 is not an outlier.
CAN A CAUSE BE ASSIGNED TO THE
OUTLIER(S) ?• Review analyst’s daily observationsReview analyst’s daily observations
• Check water chemistry dataCheck water chemistry data
• Check data entryCheck data entry
• Check calculationsCheck calculations
• If cause can be assigned to outlier, then If cause can be assigned to outlier, then reanalyze data without outlierreanalyze data without outlier
DETERMINE EFFECT ON TEST INTERPRETATION
• Keep all data unless cause is foundKeep all data unless cause is found
• Analyze data with and without suspect Analyze data with and without suspect datadata
• Determine effect of suspect data on test Determine effect of suspect data on test interpretationinterpretation
• Results reported will depend on effect of Results reported will depend on effect of outlier(s) on test interpretationoutlier(s) on test interpretation
REPORTING OF RESULTS• Insignificant EffectInsignificant Effect
– With OutlierWith Outlier• IC25 = 131 (96.9-158) ppb
• NOEC = 100 ppb• % MSD = 28.1 %
– Without OutlierWithout Outlier• IC25 = 124 (93.6-152) ppb
• NOEC = 100 ppb• % MSD = 20.9 %
• Report results with Report results with suspect data suspect data includedincluded
• Significant EffectSignificant Effect– With OutlierWith Outlier
• IC25 = 131 (96.9-158) ppb
• NOEC = 100 ppb• % MSD = 28.1 %
– Without OutlierWithout Outlier• IC25 = 106 (83.8-126) ppb
• NOEC = 50 ppb• % MSD = 12.2 %
• Report results from Report results from both analysesboth analyses
HORMESIS ANDNON-MONOTONIC CONCENTRATION
RESPONSES
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
WHAT IS HORMESIS ?
• Calabrese and Baldwin, 1998Calabrese and Baldwin, 1998
• General conceptGeneral concept
• OccurrenceOccurrence
• Typical CharacteristicsTypical Characteristics
TYPICAL TRAITS OF HORMESIS
• Hormetic - Hormetic - concentration rangeconcentration range
• Magnitude of Magnitude of hormetic stimulation hormetic stimulation
• Range from Range from maximum maximum stimulation to NOELstimulation to NOEL
Concentration
Res
pons
e
Max. Stimulation (30-60%)
Hormetic Range (10 x)
Max. Stimulationto NOEL Range
(4-5 x)
NOEL
WHY IS HORMESIS DIFFICULT TO DETECT IN TOXICITY
TESTS?• Inadequate Inadequate
concentration seriesconcentration series• Inadequate description Inadequate description
of concentration - of concentration - responseresponse
• Inadequate statistical Inadequate statistical powerpower
• Hormesis is not the Hormesis is not the causecause
Well Defined Hormetic Response
Concentration100 1000
Re
spo
nse
Poorly Defined "Hormetic" Response
Concentration100
Re
spo
nse
EFFECTS OF NON-MONOTONIC DATA
NOEC >LOECSea Urchin Fertilization Data
Percent Effluent0 1 2 3 4 5 6
Per
cent
Fer
tiliz
ed
70
75
80
85
90
95
100
Statistically Significant Reduction
NOEC = 6.0 %LOEC = 0.36 %% MSD = 5.82 %IC25 = > 6.0 %
• Limited replicates Limited replicates (4)(4)
• Control/low Control/low concentration concentration variabilityvariability
• High Statistical High Statistical PowerPower
• NOEC > LOECNOEC > LOEC
EFFECTS OF NON-MONOTONIC DATA
HETEROGENEITY IN PROBIT ANALYSIS
• Limited replicates (5)Limited replicates (5)• Control/low Control/low
concentration variabilityconcentration variability• Significant chi-square Significant chi-square • Inflated confidence Inflated confidence
intervalsintervals• Reanalyze with non-Reanalyze with non-
parametric modelsparametric models
Significant Chi-Square for Heterogeneity
0.00.10.20.30.40.50.60.70.80.91.0
1 10 100 1000 10000
Dose ppb
Re
sp
on
se
EFFECTS OF NON-MONOTONIC DATA
SMOOTHING IN ICP ANALYSIS• Smoothing is used Smoothing is used
in all non-parametric in all non-parametric models.models.
• Smoothing Smoothing procedure averages procedure averages treatment responsestreatment responses
• Increases observed Increases observed toxicitytoxicity
Selenastrum Cell Growth Data
Percent Effluent0 20 40 60 80 100
Re
spon
se (
% o
f Con
tro
l)
0
25
50
75
100
125
150
175
200
225
250
Actual ResponseSmoothed Response
REMEDIES FOR PROBLEMS ASSOCIATED WITH NON-
MONOTONIC DATA• Better concentration series selectionBetter concentration series selection• Increase number of replicatesIncrease number of replicates• % MSD limits (NOEC’s)% MSD limits (NOEC’s)• Concentration-response curve criterionConcentration-response curve criterion• Use of more robust parametric modelsUse of more robust parametric models
Bailer and Oris, 1997Bailer and Oris, 1997Kerr and Meador, 1996Kerr and Meador, 1996Baird Baird et alet al., 1996., 1996
ANALYSIS OF MULTIPLECONTROL TOXICITY TESTS
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
WHEN ARE MULTIPLE CONTROLS USED?
• To compare “standard” and “alternative To compare “standard” and “alternative methods.methods.– Food, dilution water, sterilization, organism Food, dilution water, sterilization, organism
source, source, etcetc....– Control response is often not sufficient to Control response is often not sufficient to
determine differences.determine differences.– Use of reference toxicant tests is Use of reference toxicant tests is
recommended.recommended.
EFFECT OF KELP STORAGE ON SENSITIVITY TO COPPER
Fresh Stored
Cop
per
Con
cen
trat
ion
(ppb
)
0
20
40
60
80
100
120
Effect Level1 5 10 15 25
Cha
nge
in E
CX V
alu
es
(Sto
red
- F
resh
; pp
b C
u)
-70
-60
-50
-40
-30
-20
-10
0
10EC25
*
**
*
WHY ARE MULTIPLE CONTROLS USED?
• When manipulations are made to When manipulations are made to SOMESOME of the test concentrations. of the test concentrations.– Primarily used for salinity adjustments.Primarily used for salinity adjustments.– First rule, avoid if at all possible.First rule, avoid if at all possible.– Treat extra control as most manipulated Treat extra control as most manipulated
concentration.concentration.– Purpose is to determine if adjustments Purpose is to determine if adjustments
affected test results.affected test results.
BRINE ADDITION INMARINE TESTS
Conc.
EffluentVolume(0 ppt)
BrineVolume(68 ppt)
SeawaterVolume(34 ppt) Salinity
SeawaterControl
0 ml 0 ml 1000 ml 34 ppt
0.625 % 6.25 ml 0 ml 993.75 ml 34 ppt
1.25 % 12.5 ml 0 ml 987.5 ml 34 ppt
2.5 % 25 ml 0 ml 975 ml 33 ppt
5 % 50 ml 0 ml 950 ml 32 ppt
10 % 100 ml 100 ml 800 ml 34 ppt
BrineControl
0 ml 100 ml +100 ml D.I.
800 ml 34 ppt
ANALYSIS OF TWO-CONTROL TOXICITY TESTS WHEN SOME CONCENTRATIONS
WERE MANIPULATED
N o Y es
Y esY es N o N o
A n a lyze IW C an d L ikeTrea ted C on cs . an d
C on tro l U s in gE P A F lowch arts
R ep eat Tes t
IW C Trea tedC on tro l V a lid ?
P oo l C on tro lsan d A n a lyze A ll D ata
U s in g E P A F lowch arts
A n a lyze IW C an d L ikeTrea ted C on cs . an d
C on tro l U s in gE P A F lowch arts
C on tro l t-Tes tN on -S ig n ifican t?
B oth C on tro lsV a lid ?
MOST SENSITIVE SPECIES
DETERMINATION
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
WHAT IS A MOST SENSITIVE SPECIES
SCREEN (MSSS)?
• A group of toxicity tests used to A group of toxicity tests used to determine the species/method most determine the species/method most capable of characterizing the toxicity capable of characterizing the toxicity associated with a dischargeassociated with a discharge
COMMON CONSIDERATIONS
• Test species selectionTest species selection
• Frequency/timing of initial and Frequency/timing of initial and subsequent screenssubsequent screens
• Changing effluent characteristicsChanging effluent characteristics
• Selection of data analysis methodsSelection of data analysis methods
MULTIPLE BIOLOGICAL ENDPOINT ANALYSIS
• Evaluate each Evaluate each biological endpointbiological endpoint
• Use most “toxic” Use most “toxic” endpointendpoint
Kelp Germination and Germ Tube Length
Statistical Endpoint
NOEC EC/IC25
Eff
lue
nt C
once
ntr
atio
n (%
)
0
20
40
60
80
100 GerminationTube Length
METHODS OF COMBINING MSSS RESULTS
• Proportion (X times Proportion (X times out of Y screens)out of Y screens)
• AveragingAveraging
Multiple MSSS Data Using FW Chronic Tests
Screen Number
1 2 3E
fflue
nt C
once
ntra
tion
(%)
0
20
40
60
80
100
FH CD SC
*
*
*
Species Proportion (X/Y) Average Fathead Minnow (FH) 67 % (2/3) * 87 % Ceriodaphnia (CD) 33 % (1/3) 70 % *Selenastrum (SC) 0 % (0/3) 97 %
STATISTICAL ENDPOINTS FOR EVALUATING MSSS
• NOEC’sNOEC’s
• Point-estimatesPoint-estimates
• Effect at critical concentration (ECC)Effect at critical concentration (ECC)
• Probability of effect at critical Probability of effect at critical concentration (pECC)concentration (pECC)
NOEC’S
• Experimental QuestionExperimental Question
Which method/species is Which method/species is most likely to identify a change from most likely to identify a change from
control response?control response?
ADVANTAGES OF NOEC’S
• Common endpointCommon endpoint• Integrates effect and Integrates effect and
intra-test variabilityintra-test variability
MSSS Determination Using NOEC's
Species
FH CD SC
NO
EC
(%
Effl
uent
)
0
20
40
60
80
100
*
DISADVANTAGES OF NOEC’S
• Can not separate Can not separate biological effect and biological effect and statistical sensitivitystatistical sensitivity
• Can not averageCan not average• NOEC’s may not be NOEC’s may not be
environmentally environmentally relevantrelevant
Species
FH CD SC
Eff
lue
nt C
once
ntr
atio
n (%
)
0
20
40
60
80
100
NOEC EC/IC25
>100 >100
MSSS Determination Using NOEC's
IWC
POINT-ESTIMATES
• Experimental QuestionExperimental Question
Which method/species shows Which method/species shows the specified effect at the lowest the specified effect at the lowest
concentration?concentration?
ADVANTAGES OF POINT-ESTIMATES
• Evaluates a Evaluates a common effect levelcommon effect level
• Utilizes the entire Utilizes the entire concentration-concentration-response curve response curve (parametric models)(parametric models)
• Can use proportion Can use proportion or average analysisor average analysis
MSSS Using Point-Estimates
Concentration (%)
0 20 40 60 80 100
Eff
ect
(%)
0
10
20
30
40
50
60
70
80
90
100 FH - EC/IC25 = 70 % *
CD - EC/IC25 = 90 %
SC - EC/IC25 = >100 %
DISADVANTAGES OF POINT-ESTIMATES
• Effect level selectionEffect level selection• Concentration-Concentration-
response requiredresponse required• SmoothingSmoothing• No consideration of No consideration of
endpoint precisionendpoint precision• EC values may not EC values may not
be environmentally be environmentally relevantrelevant
MSSS Using Point-Estimates
Concentration (%)0 20 40 60 80 100
Effe
ct (
%)
0
10
20
30
40
50
60
70
80
90
100 FH - EC/IC25 = 70 % *
CD - EC/IC25 = 90 %
SC - EC/IC25 = >100 %
IWC
EFFECT AT CRITICAL CONCENTRATION (ECC)• Experimental QuestionExperimental Question
Which method/species shows Which method/species shows the greatest effect at the concentration the greatest effect at the concentration
of environmental concern?of environmental concern?
ADVANTAGES OF ECC
• Can use proportion Can use proportion or average analysisor average analysis
• Environmental Environmental relevancerelevance
• No concentration-No concentration-response requiredresponse required
MSSS Using Effect at the Critical Concentration
Concentration (%)
0 20 40 60 80 100
Effe
ct (
%)
0
10
20
30
40
50
60
70
80
90
100 FH - ECC = 0 %CD - ECC = 10 % *SC - ECC = -3 %
IWC
DISADVANTAGES OF ECC
• Does not consider Does not consider certainty of certainty of response estimateresponse estimate
• Ability to obtain Ability to obtain effect estimate at effect estimate at IWC from point-IWC from point-estimate modelsestimate models
Species
FH CD SC
Effe
ct a
t Crit
ical
Con
cent
ratio
n (%
)-10
0
10
20
30
40
50
MSSS Using Effect at the Critical Concentration
PROBABILITY OF ECC (pECC)
• Experimental QuestionExperimental Question
At the concentration of At the concentration of environmental concern, which environmental concern, which
method/species had the greatest effect method/species had the greatest effect at the lower 95 % confidence limit?at the lower 95 % confidence limit?
ADVANTAGES OF pECC
Species
FH CD SC
Effe
ct (
%)
-10
0
10
20
30
ECCpECC
MSSS Using Probability of Effect at the Critical Concentration
*
• Considers precision Considers precision of response of response estimate estimate
• Can use proportion Can use proportion or average analysisor average analysis
• Environmental Environmental relevancerelevance
• No concentration-No concentration-response requiredresponse required
DISADVANTAGES OF pECC
• Zero replicate Zero replicate variance variance
• Boot-strapping Boot-strapping • Obtaining 95% Obtaining 95%
confidence intervals confidence intervals at IWCat IWC
SpeciesFH CD SC
Effe
ct (
%)
-15
-10
-5
0
5
10
ECCpECC
MSSS Using Probability of Effect at the Critical Concentration
*0 0
SUMMARY
• Discuss the MSSS procedure in detail Discuss the MSSS procedure in detail during permit developmentduring permit development
• Select variety of organism typesSelect variety of organism types• Initially test for trends in toxicityInitially test for trends in toxicity• Continue periodic screening Continue periodic screening • Select type of statistical analysis carefullySelect type of statistical analysis carefully• Make sure that statistical analysis and the Make sure that statistical analysis and the
raw results “make sense”raw results “make sense”
AGE-RELATED SENSITIVITY OF FISH IN ACUTE
WET TESTS
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
REVISIONS TO FISH AGES IN EPA ACUTE TEST
MANUALS• From: 1-90 days old in the 3rd edition From: 1-90 days old in the 3rd edition
of the acute manual (1985; EPA/600/4-of the acute manual (1985; EPA/600/4-85/013)85/013)
• To: 1-14 days old (or 9-14 days old for To: 1-14 days old (or 9-14 days old for silversides) in the 4th edition of the silversides) in the 4th edition of the acute manual (1993; EPA/600/4-acute manual (1993; EPA/600/4-90/027F)90/027F)
COMMONLY USEDTEST SPECIES
• Fathead minnowsFathead minnows
• Sheepshead minnowsSheepshead minnows
• Silversides (inland, atlantic, and Silversides (inland, atlantic, and tidewater)tidewater)
RATIONALE
• Younger life stage is generally more Younger life stage is generally more sensitive than older life stagesensitive than older life stage
• Reduction in range of acceptable ages Reduction in range of acceptable ages from 1-90 to 1-14 days will reduce from 1-90 to 1-14 days will reduce variabilityvariability
CONCERN
• Use of younger fish in NPDES testing Use of younger fish in NPDES testing may show an increase in apparent may show an increase in apparent toxicity, without any changes in effluent toxicity, without any changes in effluent conditionsconditions
COMMON QUESTIONS
• Are <14-day old fish more sensitive Are <14-day old fish more sensitive than <90-day old fish to toxicants?than <90-day old fish to toxicants?
• Does the use of <14-day old fish reduce Does the use of <14-day old fish reduce intertest variability when compared to intertest variability when compared to <90 day-old fish?<90 day-old fish?
• How does the sensitivity and precision How does the sensitivity and precision vary within the 1 to 14 day old age vary within the 1 to 14 day old age range?range?
SENSITIVITY OF 14, 30, AND 90 DAY-OLD
FATHEAD MINNOWSCopper
Age (days)14 30 90
Mea
n 96
hr
LC50
(p
pb)
0
200
400
600
800
1000
1200
Unionized Ammonia
Age (days)14 30 90
Mea
n 96
hr
LC50
(p
pm)
0.00
0.25
0.50
0.75
1.00
1.25
1.50
A
B
C
A
A
B
INTER-TEST PRECISION OF 14, 30, AND 90-Day Old
FATHEAD MINNOWSCopper
Age (days)14 30 90
Co
eff
icie
nt
of
Va
ria
tion
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Unionized Ammonia
Age (days)14 30 90
Co
eff
icie
nt
of
Va
ria
tion
0.00
0.05
0.10
0.15
0.20
0.25
SENSITIVITY OF 1-14 DAY-OLD
FATHEAD MINNOWSSodium Pentachlorophenol
Age (days)1 4 7 10 14
Mea
n 48
hr
LC50
(pp
b)
0
100
200
300
400
Hexavalent Chromium
Age (days)1 4 7 10 14
Mea
n 48
hr
LC50
(pp
m)
0
50
100
150
200
250
SDS
Age (days)1 4 7 10 14
Mea
n 48
hr
LC50
(pp
m)
01234567
Unionized Ammonia
Age (days)1 4 7 10 14
Mea
n 48
hr
LC50
(pp
m)
0.0
0.5
1.0
1.5
2.0
2.5
A
BB B B
A
AA A A
A
A
B BB B
BB
BB
INTER-TEST PRECISIONOF 1-14 DAY-OLD
FATHEAD MINNOWS
Age Range (days)1 - 14 4 - 14 7 - 14 10 - 14
Co
effic
ient
of V
aria
tion
0.0
0.1
0.2
0.3
0.4
0.5
0.6NaPCP
Cr+6
SDSNH3
SUMMARY
• 14-day old fathead minnow larvae are 14-day old fathead minnow larvae are more sensitive to copper & ammonia more sensitive to copper & ammonia than 90 day- old fish.than 90 day- old fish.
• The inter-test precision of 90 day old The inter-test precision of 90 day old fish is equal or better than 14 day-old fish is equal or better than 14 day-old fish for copper & ammonia.fish for copper & ammonia.
SUMMARY(CONTINUED)
• Within the 1-14 day age range, 1 day-Within the 1-14 day age range, 1 day-old larvae are less sensitive to several old larvae are less sensitive to several toxicants.toxicants.
• The sensitivity of these toxicants The sensitivity of these toxicants becomes constant after 4-7 days of age.becomes constant after 4-7 days of age.
• Maximum inter-test precision for these Maximum inter-test precision for these toxicants is observed when the age toxicants is observed when the age range is limited to 7 -14 day old larvae.range is limited to 7 -14 day old larvae.
THE CHRONIC TEST GROWTH ENDPOINT
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
CHANGE IN GROWTH ENDPOINT CALCULATION
Pre-Nov., 1995 ApproachPre-Nov., 1995 Approach
Growth = Growth = D.W. surviving organismsD.W. surviving organisms
# surviving organisms# surviving organisms
Post-Nov., 1995 ApproachPost-Nov., 1995 Approach
Growth = Growth = D.W. surviving organismsD.W. surviving organisms
# initial organisms# initial organisms
EFFECT ON MEAN TREATMENT RESPONSES
Treatment %Mortality
BeforePromulgation
AfterPromulgation
Control 5.1 325 308
2 2.6 353 341
3 5.0 345 329
4 17.9 387 306
5 47.5 319 167
INTRA-TREATMENT VARIABILITY AND WEIGHT
CALCULATIONS
5
10
15
20
25
30
35
Observations
CV
(%
)
After
Before
OLD MSE/NEW MSE RATIO
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1 2 3 4 5 6 7 8 9 10
Tests
Ref. Tox.
Effluent
EFFECTS ON HYPOTHESIS TEST ENDPOINTS
BeforePromulgation
AfterPromulgation
Test #
%MSD NOEC %MSD NOEC
1 16.4 50 16.7 50
2 10.8 10 29.1 10
3 11.9 5 39.0 5
4 19.7 25 18.5 25
EFFECTS ON HYPOTHESIS TEST ENDPOINTSBefore Promulgation After Promulgation
Test # %MSD NOEC Avg.wgt. atNOEC
%MSD NOEC Avg.wgt. atNOEC
1 20.9 100 296 23.4 100 296
2 19.5 100 268 25.1 100 233
3 22.1 100 254 24.1 100 227
4 21.4 100 387 22.8 100 313
EFFECTS ON POINT ESTIMATE ENDPOINTS
BeforePromulgation
AfterPromulgation
Test #
IC25 95%CI IC25 95%CI
1 56.2 45.4-79.3 48.3 43.3-61.9
2 NC NC 12.4 6.4-13.8
3 NC NC 4.2 1.5-7.3
4 33.7 28.2-40.6 30.0 19.4-35.0
EFFECTS ON POINT ESTIMATE ENDPOINTS
BeforePromulgation
AfterPromulgation
Test #
IC25 95%CI IC25 95%CI
1 291 NC 234 191-262
2 386 NC 176 140-256
3 227 179-258 138 111-155
4 >400 NC 144 104-162
NOEC/IC25 RELATIONSHIP
Test # TestType
NOEC IC25Before
IC25After
1 Effluent 50% 56.2 48.3
2 Effluent 25% 33.7 30.0
3 Ref. Tox. 100 ppb 291 234
4 Ref. Tox. 100 ppb 386 176
5 Ref. Tox. 100 ppb 227 138
6 Ref. Tox. 100 ppb >400 144
IMPACT ON TEST INTERPRETATION
• Hypothesis Test Results - most cases Hypothesis Test Results - most cases show little change, but not alwaysshow little change, but not always
• Point Estimate Results - usually Point Estimate Results - usually increases predicted toxicityincreases predicted toxicity
ISSUES RELATED TO CHANGE IN APPROACH
• Test growth or biomass?Test growth or biomass?
• Accurate representation of growth?Accurate representation of growth?
• Correlation between new results and Correlation between new results and instream responses?instream responses?
ISSUES RELATED TO CHANGE IN APPROACH
• Conflict between new results and Conflict between new results and unchanged effluent quality?unchanged effluent quality?
• Effect on reference toxicant control Effect on reference toxicant control chartscharts
• Relationship between NOEC and IC25Relationship between NOEC and IC25
ANOMALOUS PATTERNS OF SURVIVAL IN SHORT-TERM
CHRONIC WET TESTS WITH FATHEAD MINNOWS
SETAC Expert Advisory PanelSETAC Expert Advisory Panel
Performance Evaluation andPerformance Evaluation and
Data InterpretationData Interpretation
WHERE HAS THE “PROBLEM” BEEN SHOWN?• Effluent toxicity tests where receiving Effluent toxicity tests where receiving
water is used as test dilution water water is used as test dilution water (diluent)(diluent)
• ““Once through” cooling waters, where Once through” cooling waters, where receiving water is used for cooling and receiving water is used for cooling and then dischargedthen discharged
• Ambient toxicity testsAmbient toxicity tests
COMMON CHARACTERISTICS
• Observed in fathead minnow short-term chronic WET Observed in fathead minnow short-term chronic WET tests, but not in acute WET teststests, but not in acute WET tests
• Not observed in concurrently performed Not observed in concurrently performed ceriodaphniaceriodaphnia short-term chronic testsshort-term chronic tests
• High variability in survival between replicates within a High variability in survival between replicates within a concentrationconcentration
• Concentration-effects relationship is often non-Concentration-effects relationship is often non-monotonicmonotonic
• Mortality is often first seen on day 4 of the test in RW Mortality is often first seen on day 4 of the test in RW controlscontrols
INDUSTRIAL EFFLUENT #1SURVIVAL ENDPOINT (7 D)
Survival vs. Effluent Concentration
0
20
40
60
80
100
120
0 11 22 43 72 100
Effluent Concentration (%)
7-d
ay
Su
rviv
al (
%)
Laboratory water Receiving water
Coefficient of Variation vs. Effluent Concentration
05
101520
2530354045
0 11 22 43 72 100
Effluent Concentration (%)
Co
eff
icie
nt
of
Va
ria
tio
n
(%)
Laboratory water Receiving water
INDUSTRIAL EFFLUENT #1GROWTH ENDPOINT
Growth vs. Effluent Concentration
0
0.1
0.2
0.3
0.4
0.5
0.6
0 11 22 43 72 100
Effluent Concentration (%)
7-d
ay
Gro
wth
(m
g/f
ish
)
Laboratory water Receiving water
Coefficient of Variation vs. Effluent Concentration
05
101520253035404550
0 11 22 43 72 100
Effluent Concentration (%)
Co
eff
icie
nt
of
Va
ria
tio
n
(%)
Laboratory water Receiving water
INDUSTRIAL EFFLUENT #1DAILY SURVIVAL (%)
0102030405060708090
100
DA
Y 1
DA
Y 3
DA
Y 5
DA
Y 7
LAB.WATER
REC.WATER
• Majority of mortality Majority of mortality occurred on Test occurred on Test Days 3 and 4Days 3 and 4
• Final survival for Final survival for four replicates in four replicates in receiving water was receiving water was 70, 30, 40, and 70%70, 30, 40, and 70%
INDUSTRIAL EFFLUENT #2SURVIVAL ENDPOINT (7 D)
Survival vs. Effluent Concentration
0
20
40
60
80
100
120
0 6.25 12.5 25 50 100
Effluent Concentration (%)
7-d
ay
Su
rviv
al (
%)
Laboratory water Receiving water
Coefficient of Variation vs. Effluent Concentration
0
5
10
15
20
25
30
35
40
0 6.25 12.5 25 50 100
Effluent Concentration (%)
Co
eff
icie
nt
of
Va
ria
tio
n
(%)
Laboratory water Receiving water
INDUSTRIAL EFFLUENT #2GROWTH ENDPOINT
Growth
Growth vs. Effluent Concentration
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 6.25 12.5 25 50 100
Effluent Concentration (%)
7-d
ay
Gro
wth
(m
g/f
ish
)
Laboratory water Receiving water
Coefficient of Variation vs. Effluent Concentration
0
5
10
15
20
25
0 6.25 12.5 25 50 100
Effluent Concentration (%)
Co
eff
icie
nt
of
Va
ria
tio
n
(%)
Laboratory water Receiving water
WISCONSIN DNR PROGRAM
• For chronic WET tests performed For chronic WET tests performed between 1988-1998, 26% showed between 1988-1998, 26% showed unacceptable receiving water control unacceptable receiving water control survival survival
• Only 2.9% of lab controls failed to meet Only 2.9% of lab controls failed to meet survival criterionsurvival criterion
• Effects are not seasonalEffects are not seasonal
MICROBIOLOGICAL EXAMINATION OF FISH
• Aeromonas hydrophilaAeromonas hydrophila
• Flexibacter aurantic and F. columnarisFlexibacter aurantic and F. columnaris
• Flavobacterium sp.Flavobacterium sp.
• Saprolegnia sp.Saprolegnia sp.
WHAT WORKS TO ELIMINATE THE PROBLEM?• Filtration (0.2 µ; some success with Filtration (0.2 µ; some success with
0.45µ)0.45µ)
• AutoclavingAutoclaving
• UV disinfectionUV disinfection
• HeatingHeating
• AntibioticsAntibiotics
HOW ARE PEOPLE HANDLING THIS PROBLEM?
• Use laboratory water as test dilution and Use laboratory water as test dilution and control water for all WET testingcontrol water for all WET testing
• Use laboratory water, after receiving Use laboratory water, after receiving water problems have been shownwater problems have been shown
• Perform concurrent testing in both Perform concurrent testing in both laboratory water and receiving waterlaboratory water and receiving water
HOW ARE PEOPLE HANDLING THE PROBLEM?
(CONTINUED)
• Accept “problem” tests for compliance, but don’t use Accept “problem” tests for compliance, but don’t use them to determine “pass/fail”them to determine “pass/fail”
• Manipulate receiving water sample, before use in testManipulate receiving water sample, before use in test
SUMMARY• Interpretation of fathead minnow short-term Interpretation of fathead minnow short-term
chronic tests may be complicated by presence chronic tests may be complicated by presence of a “biological agent”of a “biological agent”
• This problem has been observed in many This problem has been observed in many areas, but it is not known how widespread is its areas, but it is not known how widespread is its occurrenceoccurrence
• Phenomenon is being studied by different Phenomenon is being studied by different investigators and may result in investigators and may result in recommendations for test method modificationsrecommendations for test method modifications