natural resources defense council’s & powder river basin resource council… ·...
TRANSCRIPT
NATURAL RESOURCES DEFENSE COUNCIL’S & POWDER RIVER BASIN
RESOURCE COUNCIL’S PETITION FOR REVIEW
EXHIBIT 14
5 6 4 4
CHARACTERIZATION OF BACKGROUND WATER QUALITY
FOR STREAMS AND GROUNDWATER
FERNALD ENVIRONMENTAL MANAGEMENT PROJECT FERNALD, OHIO
REMEDIAL INVESTIGATION and FEASIBILITY STUDY
I-
May 1994 .
U.S. DEPARTMENT OF ENERGY FERNALD FIELD OFFICE
DRAFT’”FINAL 4 . , . . ,
J
I 5644 Background Study
May 1994
TABLE OF CONTENTS
List of Tables List of Figures List of Acronyms 1.0 Introduction -
1.1 Brief History of the Site 1.2 Purpose 1.3 Geologic Setting 1.4 Hydrologic Setting
1.4.1 Great Miami River 1.4.2 Paddys Run 1.4.3 Great Miami Aquifer 1.4.4 Glacial Overburden 1.4.5 Monitoring Wells
1.5 EPA Guidance on Background Characterization 1.6 Summary of Revisions Made to the Draft Report (May 1993)
2.0 Previous Studies
3.0
2.1 Environmental Monitoring Program 2.2 U.S. Geological Survey Surface Water Monitoring 2.3 U.S. Geological Survey Groundwater Study 2.4 IT Corporation Final Interim Report 2.5 Argonne National Laboratory Environmental Survey 2.6 Ohio Department of Health 2.7 Ohio Environmental Protection Agency Study of the Great Miami River 2.8 Previous RI/FS Background Studies 2.9 RCRA Groundwater Monitoring Development of the RI/FS Background Data Set 3.1 Sampling Locations
3.1.1 Surface Water, 3.1.2 Groundwater
3.1.2.1 Identification of Potential Background Monitoring Wells 3.1.2.2 Screening of Background Locations 3.1.2.3 General Water Chemistry and Charge Balance 3.1.2.4 Summary
3.2 Sample Collection 3.3 Analytical Procedures
Eiw? iv
vii viii 1-1 1-1 1-3 1-4 1-7 1-7
1-10 1-11 1-13 1-13 1-15 1-16 2-1 2-1 2-1 2-3 2-3 2-3 2-3 2-6 2-6 2-7 3-1 3-1 3-1 3-2 3-2 3-4
3-11 3-15 3-19 3-20
0USIG:BACKGRDISECS. 1-9.TOC.IOS-W i
Background Study May 1994
TABLE OF CONTENTS (continued)
Page
3.4 Data Validation Procedures 3.5 Validated and Deleted Data Data Set Modifications and Statistical Analysis Procedures 4.1 Overview 4.2 Modifications of the Background Data Set
4.2.1 Treatment of Rejected/Nonvalidated Data 4.2.2 Treatment of Nondetect Data 4.2.3 Identification and Treatment of Outliers and Other "Suspect" Data 4.2.4 Data Averaging
4.3.1 Testing of Data Distribution 4.3.2 Parametric Descriptive Statistics 4.3.3 Nonparametric Descriptive Statistics 4.3.4 Comparison of Populations
4.4 Summary of Revisions to Chapter 4 of the Draft Report
4.0
4.3 Statistical Analysis
5.0 Glacial Overburden a 5.1 Radiological Constituents 5.2 Inorganic Constituents 5.3 Organic Constituents
6.1 Radiological Constituents 6.2 Inorganic Constituents 6.3 Organic Constituents
7.1 Radiological Constituents 7.2 Inorganic Constituents 7.3 Organic Constituents
8.1 Radiological Constituents 8.2 Inorganic Constituents 8.3 Organic Constituents
References
6.0 Great Miami Aquifer
7.0 Great Miami River
8.0 Paddys Run
9.0 Conclusions
3-20 3-21 4-1 4-1 4-1 4-2 4-2 4-3, 4-5 4-5 4-5 4-7 4-7 4-8 4-8 5-1 5-1 5-2 5-3 6-1 6-1 6-2 6-3 7-1 7-1 7-1 7-2 8-1 8-1 8-1 8-2 9-1 R-1.
0USIG:BACKGRDISECS. 1-9.,W./OS-W ii
5644 . - Background Study
May 1994
TABLE OF CONTENTS (continued)
Appendix A - Data from Previous Studies Appendix B - Drilling Logs and Well Construction Information Appendix C - Radiological Data Appendix D - Inorganic Chemical Data Appendix E - Organic Chemical Data Appendix F - Statistical Procedures, Equations, and Results Appendix G - Summary of Revisions to the "Characterization of
Background Water Quality for Streams and Groundwater" Draft Report (May 1993)
Monitoring Wells in the Tributary Sections of the Great Miami Aquifer
Appendix H - Summary Statistics of Inorganic Constituents for Background
000004 OU5IG:BACKGRDISECS. 1-9.TOc./OS-W iii
Table E-19 Rejected/Nonvalidated Organic Data for Background Surface
Water in the Great Miami River
Well No. W-1 W-1 W-1
Sample lab Validated . QA Date ID qualifier Constituent Result Qualifier type
05/20/93 120064-2 U 4-Nitroaniline 25 R N 05/20/93 120068-1 U 4-Nitroaniline 25 R D 05/20/93 120072-2 U 4-Nitroaniline 25 R T
, .. . . . .
E-210
c'
Well Sample lab Validated QA No. Date ID qualifier Constituent Result Qualifier type W-5 03/25/93 113493 U 2,4-Dinitrophenol 50 R N
I W-5 03/25/93 113493 U 4,6-Dinitro-2-methylphenol 25 R N
Table E-20 Rejected/Nonvalidated Organic Data for Background Surface
Water in Paddys Run
E-21 1
d.
APPENDIX F
STATISTICAL PROCEDURES, EQUATIONS, AND RESULTS
TABLE OF CONTENTS
List of Tables Shapiro-W& Test for Normality Shapiro-Francia Test for Normality Rosner's Test for Many Outliers Data Averaging Sample Arithmetic Mean - Normal Distriiution Sample Arithmetic Standard deviation - Normal Distribution Estimated Coefficient of Variation - Normal Distribution Estimated Mean of a Lognormal Distribution Estimated Standard Deviation of a Lognormal Distribution Sample Median - Nonparametric Technique Upper One-Sided 95% Confidence Limit - Normal Distribution Upper One-sided 95% Confidence Limit - Lognormal Distribution Upper One-sided 95% Confidence Limit - Nonparametric Technique 95'h Percentile - Normal Distribution 9Sth Percentile - Lognormal Distribution 95'h Percentile - Nonparametric Technique F-Test T-Test The Wilcoxin Rank Sum Test Kruskal-Wallis Test
F-ii F-5
F-14 F-23 F-30 F-33 F-34 F-35 F-36 F-38 F-39 F-40 F-43 F-47 F-49 F-50 F-51 F-52 F-57 F-60 F-64
F-i
000924
LIST OF TABLES
F- 1
F-2
F-3
F-4
F-5
F-6
F-7
F-8
F-9
F-10
F-11
F-12
F-13
F-14
F-15
F-16
F-17
F-18
Formulas for Summary Statistics
Coefficients 3 for the Shapiro-Wilk W Test for Normality
F- 1
F-7
Quantiles of the Shapiro-Wilk W Test for Normality (Values of W Such that 100 p% of the Distribution of W Is Less Than WJ
Example Data Set Number 1
Standard Normal Curve for a Z Distribution
F-9
F-11
F-16
Percentage Points of the W' Test for n > 50
Example Data Set Number 2
F-18
F-2 1
Approximate Critical Values Lamda (it 1 ) for Rosner's Generalized ESD Many-Outlier Procedure for alpha = 0.05
Example Data Set Number 3
Example Data Set Number 4
F-25
F-27
F-3 1
Quantiles of the t Distribution (Values o f t Such that 100 p%
Values of H (1-alpha) for Computing One-sided (Upper) 95% Confidence
of the Distribution Is Less Than 5) F-4 1
Limits on a Lognormal Mean F-44
Percentage Points of the F Distribution (Fo.025,dm,dn)
Example Data Set Number 5
F-53
F-55
Example Data Set Number 6 F-62
Quantities of the Chi-square Distribution With v Degrees of Freedom
Ranking of Example Data Set Number 7
Ranking of Example Data Set Number 7 By Group
F-67
F-68
F-69
F-ii
000925
FEMP Background Study . May 1
Table F-1 Formulas for Summary Statistics
statistic
ShapiiW& Test (Gilbert 1987, Equations 12.3 and 12.4)
ShaphFrancia Test (Shapiro-Francia, 19tz)
_ _ ~
Rosner's Test for Many Outliers (Gilbert 1987, Equations 15.1 to 15.3)
Sample Arithmetic Mean (Gilbert 1987, Equation 43)
Formula
2
where: n n 2
d E C xi2 -- i = l f,P i = l 4
n
- n - 1
k =; i fn i sevcn
= i fn i sodd a, = Shapiro-Wilk coefficient x, = ith data value in the ordered data set
n = numberofdatapoints W = Shapim-Wilk test statistic
= square of the ith data value in the ordered data set
where:
mi = normal qwntile a-1 = inverse of standard normal distribution x, = ith data value in the ordered data set s2 = sample arithmetic variance W' = Shapiro-Francia test statistic
See page F-23.
.'--cy I n
i.1
where: n = number of data points x, = i* data value in the ordered data set x = arithmetic mean -
F- 1
64% TABLE F-1 (Continued)
FEMP Background Study May 1994
Statistic
Sample Arithmetic Standard Deviation (Gilbert 1987, Equation 4.4)
Estimated Coefficient of Variation (Gilbert 1987, Page 34)
Estimated Mean of a Lognormal Distribution (Gilbert 1987, Equation 13.7)
Estimated Standard Deviation of a Lognormal Distribution (Gilbert 1987, Equation 13.8)
Formula
whcm: n = number of data points
x = arithmetic mean s2 = arithmetic variance s = arithmetic standard deviation
3 = data Set value
CV - s f i
where: - x = sample arithmetic mean s = sample arithmetic standard deviation CV = estimated coefficient of variation
ji - exp [. + $1 where:
6y = arithmetic standard deviation of the In transformed data
p = estimated mean of a lognormal distribution
- y = arithmetic mean of the In transformed data
A
b - /b' [ exp s; - 11
where: A = estimated mean of a lognormal distribution P 6y = arithmetic standard deviation of the In transformed data
= estimated standard deviation of the lognormal distribution
000927
F-2 * . n
r.'.., -... 3 . . . ' 2
statistic
Simple Median - Nonparametric Technique (Gilbert 1987, Equation 13.15 and 13.16)
~ ~~~
Upper 95% Confidence Limit on the Mean - Normal Distribution (Gilbert 1987, Equation 11.6)
Upper 95% Confidence Limit on the Arithmetic Mean for Lognormal Distribution (Gilbert 1987, Equation 13.13)
TABLE F-1 (Continued)
Formula
I f n i s o d d
sample median = xI(" .
If n is cvcn:
w h C X
xm = i" data value in the ordered data set
n = number of data points
whem - x = arithmetic mean t o.95,n-, = student t distribution value n = number of data points s = arithmetic standard deviation 95% UCL, = one-sided upper 95% confidence limit for a normal
distribution
where: y = arithmetic mean of the In transformed data sy2 = arithmetic variance of the In transformed data sy = arithmetic standard deviation of the In transformed data
b.95 = value used to compute one-sided 95% confidence limit on a lognormal mean
n = number of data points 95% U q = one-sided upper 95% confidence limit for a lognormal
-
distribution
000928 . .
F-3
PeMP Background Study May 1994
TABLE F-1 (Continued)
statistic
Upper 95% Confidence Limit on the Median - Nonparametric Technique (Gilbert 1987, Equation 1322)
statistic
Upper 95% Confidence Limit on the Median - Nonparametric Technique (Gilbert 1987, Equation 1322)
0-95 Quantitle - Nonnal Distribution (Gilbert 1987. Equation 11.1)
0.95 Quantile - Lognormal Distribution (Gilbert 1987, Equation 13.24)
0.95 Quantilc - Nonparametric Technique
P t a t
T-tat
Wilcoxon Rank Sum t a t
hka l -Wal l i s test
Formula
wbelC: n = number of data points %.95 = upper 95% limit from a standard normal curve for a Z distribution U = rank in an ascending order data set that corresponds to the one-sided
f(U) = U rounded up to an integer (e.&, 242 -. 25) 95% UCL,.,, = data point in the ascending ordered data set at rank f(u)
upper 95% confidence limit for nonparametric distribution *.
where: - x = arithmetic mean s = arithmetic standard deviation &,g5 = 0.95 limit from a standard normal curve for a Z distribution 95" PercentileN = 9Sth percentile for normal distribution
where: 7 = arithmetic mean of the In transformed data 5 = arithmetic standard deviation of the In transformed data
Z& = upper 95% limit from a standard normal curve for a distribution 9Sth PercentileL = 9Sth percentile for lognormal distribution
Q = 0.95 n
sth PercentileNp - x ~ ~ ( ~ ) ~
where: n = number of data points Q ' = rank in an ascending order data set that corresponds to the 95"
percentile based on a nonparametric technique f(Q) = Q rounded up to an integer (e.& 14.1 4 1s) 95" PercentileNp = data point in the ascending order data bet at rank f(Q)
See WEC F-52.
See pa@ F-57.
Set page F-60.
See page F-64.
F-4 (900929
FEW Background Study May 1994
Shapiro-Wilk Test for Normality
The W test developed by Shapiro and Wilk (Gilbert 1987, Equations 12.3 and 12.4) was used to determine whether or not a data set has been drawn from a population which is normally distributed for sample size of 50 or less. By conducting this test on the natural logarithm of each data value, the W test was used to determine whether or not the sample was drawn from an underlying lognormal distribution. The null hypothesis to be tested is:
The population has a normal (lognormal when the data is transformed) distribution.
versus
H A : The population does not have a normal (lognormal when the data is transformed) distribution.
If H, is rejected, then HA is accepted.
The following presents a step-by-step procedure for conducting the W test. The equation for calculating W is:
l 2 k
1. Compute the denominator (d) of the W test statistic
2 n n
i=l
F-5 000930
FEW Background Study May 1994
where:
n c x, = x, + x2 + .... + xn
i= l
n
i=l 2 2 c 4 = x i 2 +x2 + ... + d
2. Order the n data points in ascending order (smallest to largest) such that x, - x2 5 x3 I...< q
3. Compute k, where:
n 2
k = - i f n is even
n - 1 2
k = - i f n is odd
4. Find the coefficients a,, +, a,, ..., ak for the sample size n from Table F-2.
5. Compute W
1 w = - d
6. Reject H,, at the a significance level if W is less than the quantile given in Table F-3.
To test the null hypothesis that the population has a lognormal distribution, transform the observed data to y,, y2, ..., yn where yi = In 3. Repeat steps 1 through 6 as described above.
F-6
000931
I i
3 4 5 6 7 8 9 0.7071 0.6872 0.6646 0.6431 0.6233 0.6052 0.5888 O.OOO0 0.1677 0.2413 0.2806 0.3031 0.3164 0.3244
0.0000 0.0875 0.1401 0.1743 0.1976 O.oo00 0.0561 0.0947
0.0000
- May 1994
10 0.5739 0.3291 0.2141 0.1224 0.0399
4 -
19 0.4808 0.3232 0.2561 0.2059 0.1641 0.1271 0.0932 0.061 2 0.0303 0.0000
E 1 2 3 4
' 5 6 7
20 0.4734 0.321 1 0.2565 0.2085 0.1686 0.1334 0.1013 0.071 1 0.0422 0.0140
E 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -
15 0.51 50 0.3306 0.2495 0.1878 0.1353 0.0880 0.0433 0.0000
11 0.5601 0.331 5 0.2260 0.1429 0.0695 0.0000
I
16 0.5056 0.3290 0.2521 0.1939 0.1447 0.1005 0.0593 0.0196
13 0.5359 0.3325 0.2412 0.1707 0.1099 0.0539 0.0000
12 0.5475 0.3325 0.2347 0.1586
0.0303 1 0.0922
14 0.5251 0.3318 0.2460 0.1802 0.1240 0.0727 0.0240
~
28 0.4328 0.2992 0.2510 0.2151 0.1857 0.1601 0.1372 0.1162 0.0965 0.0778 0.0598 0.0424 0.0253 0.0084
~
29 30 0.4291 0.4254 0.2968 0.2944 0.2499 0.2487 0.2150 0.2148 0.1864 0.1870 0.1616 0.1630 0.1395 0.1415 0.1192 0.1219 0.1002 0.1036 0.0822 0.0862 0.0650 0.0697 0.0483 0.0537 0.0320 0.0381 0.0159 0.0227 0.0000 0.0076
24 0.4493 0.3098 0.2554 0.2145 0.1807 0.1512 0.1245 0.0997 0.0764 0.0539 0.0321 0.0107
25 0.4450 0.3069 0.2543 0.2148 0.1822 0.1539 0.1283 0.1046 0.0823 0.0610 0.0403 0.0200 0.0000
26 0.4407 0.3043 0.2533 0.21 51 0.1836 0.1563 0.1316 0.1089 0.0876 0.0672 0.0476 0.0284 0.0094
0.3185 0.2578 0.21 19 0.1736 0.1399 0.1092 0.0804 0.0530 0.0263 0.0000
27 0.4366 0.3018 0.2522 0.21 52 0.1848 0.1584 0.1346 0.1 128 0.0923 0.0728 0.0540 0.0358 0.01 78 0.0000
0.3156 0.3126 0.2571 0.2563 0.2131 0.2139 0.1764 0.1787 0.1443 0.1480 0.1150 0.1201 0.0878 0.0941 0.0618 0.0696 0.0368 0.0459 0.0122 0.0228
0.0000
000932 F-7
E 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 - Ei
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 -
31 0.4220 0.2921 0.2475 0.2145 0.1 874 0.1641 0.1433 0.1243 0.1066 0.0899 0.0739 0.0585 0.0435 0.0289 0.0144 0.0000
41 0.3940 0.271 9 0.2357 0.2091 0.1876 0.1693 0.1531 0.1384 0.1249 0.1 123 0.1004 0.0891 0.0782 0.0677 0.0575 0.0476 0.0379 0.0283 0.0188 0.0094 0.0000
32 0.41 88 0.2898 0.2462 0.2141 0.1878 0.1651 0.1449 0.1265 0.1093 0.0931 0.0777 0.0629 0.0485 0.0344 0.0206 0.0068
42 0.391 7 0.2701 0.2345 0.2085 0.1874 0.1694 0.1535 0.1392 0.1259 0.1 136 0.1020 0.0909 0.0804 0.0701 0.0602 0.0506 0.041 1 0.031 8 0.0227 0.0136 0.0045
Table F-2 (Continued) Coefficients ai for the ShapireWilk W Test for Normality
33 0.4156 0.2876 0.2451 0.21 37 0.1880 0.1660 0.1463 0.1284 0.1 116 0.0961 0.081 2 0.0669 0.0530 0.0395 0.0262 0.01 31 0.0000
43 0.3894 0.2684 0.2334 0.2078 0.1871 0.1695 0.1539 0.1398 0.1269 0.1149 0.1035 0.0927 0.0824 0.0724 0.0628 0.0534 0.0442 0.0352 0.0263 0.01 75 0.0087 0.0000
source: Table A-6, Gilbert 1987.
. . * / , I. ., , Ui' u ' '
CIN/OU5RI /419195/TABF2.XLS/5-94
34 0.4127 0.2854 0.2439 0.2132 0.1882 0.1667 0.1475 0.1301 0.1140 0.0988 0.0844 0.0706 0.0572 0.0441 0.0314 0.0187 0.0062
-
- 44
0.3872 0.2667 0.2323 0.2072 0.1868 0.1695 0.1542 0.1405 0.1278 0.1 160 0.1049 0.0943 0.0842 0.0745 0.0651 0.0560 0.0471 0.0383 0.0296 0.021 1 0.01 26 0.0042
35 0.4096 0.2834 0.2427 0.21 27 0.1883 0.1673 0.1487 0.1317 0.1 160 0.1013 0.0873 0.0739 0.061 0 0.0484 0.0361 0.0239 0.01 19 0.0000
45 0.3850 0.2651 0.2313 0.2065 0.1865 0.1695 0.1545 0.1410 0.1286 0.1170 0.1062 0.0959 0.0860 0.0765 0.0673 0.0584 0.0497 0.041 2 0.0328 0.0245 0.0163 0.0081 0.0000
F-8
36 0.4068 0.2813 0.2415 0.2121 0.1883 0.1678 0.1496 0.1331 0.1179 0.1036 0.0900 0.0770 0.0645 0.0523 0.0404 0.0287 0.01 72 0.0057
46 0.3830 0.2635 0.2302 0.2058 0.1862 0.1695 0.1548 0.1415 0.1293 0.1 180 0.1073 0.0972 0.0876 0.0783 0.0694 0.0607 0.0522 0.0439 0.0357 0.0277 0.0197 0.01 18 0.0039
37 0.4040 0.2794 0.2403 0.21 16 0.1883 0.1683 0.1505 0.1344 0.1 196 0.1056 0.0924 0.0798 0.0677 0.0559 0.0444 0.0331 0.0220 0.01 10 0.0000
47 0.3808 0.2620 0.2291 0.2052 0.1859 0.1695 0.1550 0.1420 0.1300 0.1 189 0.1085 0.0986 0.0892 0.0801 0.071 3 0.0628 0.0546 0.0465 0.0385 0.0307 0.0229 0.0153 0.0076 0.0000
FEMP Background Study May 1994
38 0.401 5 0.2774 0.2391 0.21 10 0.1881 0.1686 0.1513 0.1356 0.121 1 0.1075 0.0947 0.0824 0.0706 0.0592 0.0481 0.0372 0.0264 0.0158 0.0053
48 0.3789 0.2604 0.2281 0.2045 0.1855 0.1693 0.1551 0.1423 0.1306 0.1 197 0.1095 0.0998 0.0906 0.081 7 0.0731 0.0648 0.0568 0.0489 0.041 1 0.0335 0.0259 0.0185 0.01 11 0.0037
-
39 0.3989 0.2755 0.2380 0.21 04 0.1880 0.1689 0.1520 0.1366 0.1225 0.1092 0.0967 0.0848 0.0733 0.0622 0.051 5 0.4090 0.0305 0.0203 0.0101 0.0000
49 0.3770 0.2589 0.2271 0.2038 0.1851 0.1692 0.1553 0.1427 0.1312 0.1205 0.1105 0.1010 0.091 9 0.0832 0.0748 0.0667 0.0588 0.051 1 0.0436 0.0361 0.0288 0.0215 0.0143 0.0071 0.0000
40 0.3964 0.2737 0.2368 0.2098 0.1878 0.1691 0.1526 0.1376 0.1237 0.1 108 0.0986 0.0870 0.0759 0.0651 0.0546 0.0444 0.0343 0.0244 0.0146 0.0049
50 0.3751 0.2574 0.2260 0.2032 0.1847 0.1691 0.1554 0.1430 0.1317 0.1212 0.1 113 0.1020 0.0932 0.0846 0.0764 0.0685 0.0608 0.0532 0.0459 0.0386 0.031 4 0.0244 0.01 74 0.01 04 0.0035
000935
f 5644
FEMp Background Study May 1994
Table F-3 Quantiles of the Shapiro-Wdk W Test for Normality
(Values of W Such That 100 p% of the Distribution of W Is Less Than W,,)
n
3 4 5 6 7 8 9 10 11 12 13 14 1s 16 17 18 19 20 21 22 23 24 25 24 27 28 29 30 31 32 33 34 35 36 37 38 3? 40 41 42 43 44 45 46 47 48 49 50
wo 01
0.753 0.687
' -. 0.686
0.m 0.749 0.764 0.781 0.792 0.805 0.814 0.825 0.835 0.844 O S 1 0.858 0.863 0.868 0.873 0.878 0.881 0.884 0.886 0.891 0.894 0.8% 0.898 0.900 0.902 0.904 0.906 0.908 0.910 0.912 0.914 0.916 0.917 0.919 0.920 0.922 0.923 0.924 0.926 0.927 0.928 0.929 0.929 0.930
0.713
wo 0.2
0.7% 0.707 0.715 0.743 0.760 0.78 0.791 0.806 0.817 0.828 0.837 0.846 O S 5 0.863 0.869 0.874 0.879 0.884 0.888 0.892 0.8% 0.898 0.901 0.904 0.906 0.908 0.910 0.912 0.914 0.91s 0.917 0.919 0.920 0.922 0.924 0.925 0.927 0.928 0.929 0.930 0.932 0.63 0.934 0.935 0.936 0.937 0.937 0.938
Wo 05
0.767 0.748 0.762 0.788 0.803 0.818 0.829 0.842 0.850 0.859 0.866 0.874 0.881 0.887 0.892 0.897 0.901
, 0.905 0.908 0.911 0.914 0.916 0.918 0.920 0.923 0.924 0.926 0.927 0.929 0.930 0.931 0.933 0.934 0.935 0.936 0.938 0.939 0.940 0.941 0.942 0.943 0.944 0.945 0.945 0.946 0.947 0.947 0.947
wo 0.789 0.792 0.806 0.826 0.838 0.851 0.859 0.869 0.876 0.883 0.889 0.895 0.901 0.906 0.910 0.914 0.917 0.920 0.923 0.926 0.928 0.930 0.931 0.933 0.935 0.936 0.937 0.939 0.940 0.941 0.942 0.943 0.944 0.945 -
0.946 0.947 0.948 0.949 0.950 0.951 0.951 0.952 0.953 0.953 0.954 0.954 0.955 0.955
woso 0.959 0.935 0.927 0.927 0.928 0.932 0.935 0.938 0.940 0.943 0.945 0.947 0.950 0.952 0.954 0.956 0.957 0.959 0.960 0.961 0.%2 0.963 0.964 0.965 0.965 0.966 0.966 0.967 0.967 0.968 0.968 0.969 0.969 0.970 0.970 0.971 0.971 0.971 0.972 0.972 0.973 0.973 0.973 0.974 0.974 0.974 0.974 0.974
Source: Table A-7, Gilbcrt 1987
F-9 000934
. , '.. i. ', 1
FEMP Background Study May 1994
Example:
To illustrate the application of the Shapiro-Wilk test, the data in Table F-4 is used.
1. Compute d, the denominator of the W test.
2 n n
i= l
1 36
= 0.0808 - - (1.634)*
= 0.0066
2. Order data points from low to high.
3. Compute k for n = 36
4. Find coefficients a,, a,, ..., aU for n = 36 from Table F-2.
a, = 0.4068 a, = 0.1678 a,, = 0.0900 a16 = 0.0287 a, = 0.2813 a, = 0.1496 a,, = 0.0776' a,, = 0.0172 a, = 0.2415 a, = 0.1331 a13 = 0.0645 a18 = 0.0057 a, = 0.2121 a, = 0.1179 al, = 0.0523 a, = 0.1883 a,, = 0.1036 aU = 0.0404
000935
. 5644
Chemical Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Total
FEMP Background Study May 1994
Table F-4 Example Data Set Number 1
Validated result,
0.034 0.049 0.05 0.05 0.05 0.05 0.064 0.035 0.035 0.039 0.04 0.04 0.04 0.04 0.043 0.044 0.044 0.045 0.045 0.047 0.048 0.049 0.05 0.051 4 0.052 0.054 0.055 0.057 0.06 0.06 0.061 0.061 0.062 0.064 0.066 0.073
mg/L
Concentration used Midation qualifier UJ U U U U U U
J J
J
J
J J J
J
J
in stat1 Normal 0.01 7 0.0245 0.025 0.025 0.025 0.025 0.032 0.035 0.035 0.039 0.04 0.04 0.04 0.04 0.043 0.044 0.044 0.045 0.045 0.047 0.048 0.049 0.05 0.051 4 0.052 0.054 0.055 0.057 0.06 0.06 0.061 0.061 0.062 0.064 0.066 0.073 1.634
1 Lognormal -4.075 -3.709 -3.609 -3.689 -3.609 -3.689 -3.442 -3.352 -3.352 -3.244 -3.219 -3.21 9 -3.219 -3.219 -3.147 -3.124 -3.124 -3.101 -3.101 -3.058 -3.037 -3.01 6 -2.996 -2.968 -2.957 -2.91 9 -2.900 -2.865 -2.81 3 -2.81 3 -2.797 -2.797 -2.781 -2.749 -2.71 8 -2.61 7
-1 13.201
000936 F-1 1
FEMP Background Study May 1994
5. Compute W.
1 d
= - [0.4068 (0.073 - 0.017) + 0.2813 (0.066 - 0.0245) + 0.2415 (0.064 - 0.025)
+ 0.2121 (0.062 - 0.025) + 0.1883 (0.061 - 0.025) + 0.1678 (0.061 - 0.025)
+ 0.1496 (0.060 - 0.032) + 0.1331 (0.06 - 0.035) + 0.1179 (0.057 - 0.035)
+ 0.1036 (0.055 - 0.039) + 0.0900 (0.054 - 0.04) + 0.0770 (0.052 - 0.04)
+ 0.0645 (0.0514 - 0.040) + 0.0523 (0.050 - 0.040) + 0.0404 (0.049 - 0.043)
+ 0.0287 (0.048 - 0.044) + 0.0172 (0.047 - 0.044) + 0.0057 (0.045 - 0.045)12
- (0.080)2
= 0.971
- 0.0066
6. Fail to reject H, at the 0.05 significance level because W,, = 0.971 is greater than Wcritical of 0.935, the quantile given in Table F-3, and conclude that the data were drawn from a population with an underlying normal distribution.
The W test was repeated after the transformation of yi = In resulted in W,,. = 0.928. Because the calculated value is less than the critical value (WrritiFl = 0.935), there is sufficient evidence to reject the null hypothesis. The conclusion is that the data set was not drawn from a population having an underlying lognormal distribution.
When the W test fails to reject the null hypotheses for both the normal and lognormal distributions, the W,,, value which exceeds the Wrritid value the most is selected as the distribution for the data set. When the .W test rejects the null hypotheses for both the
5644 FEMP Background Study
May 1994
normal and lognormal distribution but one is close (W,,, > 0.95 Wcritia,) to the test value, then this distribution (normal or lognormal) is selected. If both the normal and lognormal W,,, values are not close to Wcritia,, then the distribution is considered to be undefined and nonparametric statistical analyses will be used to described the data set.
F-13 000938
'. .
. '. , .. : .
a FEW Background Study May 1994
Shapiro-Francia Test for Normality
The W test developec by Shapiro and Wilk was used to determine u,ether or not a data set has been drawn from a population which is normally distributed for sample size of 50 or less while the Shapiro-Francia test (Shapiro and Francia, 1972) was used when the sample size was greater than 50.
Like the Shapiro-Wilk test, the Shapiro-Francia test statistic (Wl) can be calculated using the natural logarithm of each data value. This is used to determine whether or not the sample was drawn from an underlying lognormal distribution. The null hypothesis to be tested is:
H,: The population has normal (lognormal when data is transformed) distribution.
versus
HA: The population does not have a normal (lognormal when data is transformed) distribution.
If H, is rejected, then HA is accepted.
To calculate the test statistic one can use the following formula:
where represents the ith ordered value of the sample and where m, denotes the approximate expected value of the ith ordered normal quantile. The value for m, can be approximately computed as:
F-14
5644
FEW Background Study May 1994
where @-' denotes the inverse of the standard normal distribution with zero mean and unit variance. These values can be computed by hand using a normal probability table (Table F-5) or via simple commands in many statistical computer packages.
1. Order the n data points in assending order (smallest to largest) such that x1 <x2 < x3 < ... e%. a, - - - -
2. Computer [i/(n+ l)] when n is the total number of samples and i is the ith data point.
3. Compute the normal quantile (mi).
4. Compute the square of the normal quantile.
5. Compute the multiplication of the normal quantile times the corresponding data value.
6. Compute the arithmetic standard deviation.
n
i=l c (3 - 3 2
n-1 s =
where:
7. Compute the denominator of Shapiro-Francia Test.
8. Compute the numerator of the Shapiro-Francia Test.
9. Compute W'.
10. Reject H, at the Q significance level is W I is less than the quantile given in Table F-6.
F-15
FEMP Background Study May 1994
z -3.4 -3.3 -3.2 -3.1 -3.0
-2.9 -2.8 -2.7 -2.6 -2.5
-2.4 -2.3 -2.2 -2.1 -2.0
-1.9 -1.8 -1.7 -1.6 -1.5
-1.4 -1.3 -1.2 -1.1 -1.0
-0.9 -0.8 -0.7 -0.6 -0.5
-0.4 -0.3 -0.2 -0.1 -0.0
0.0 0.1 0.2 0.3 0.4
0.5 0.6 0.7 0.8 0.9
p
Table F-5 Standard Normal Curve for a 2 Distribution
0
0.0003 0.0005 0.0007 0.0010 0.0013
0.0019 0.0026 0.0035 0.0047 0.0062
0.0082 0.0107 0.0139 0.0179 0.0228
0.0287 0.0359 0.0446 0.0548 0.0668
0.0808 0.0968 0.1151 0.1357 0.1587
0.1841 0.2119 0.2420 0.2743 0.3085
0.3446 0.3821 0.4207 0.4602 0.5000
0.5000 0.5398 0.5793 0.6179 0.6554
0.6915 0.7257 0.7580 0.7881 0.8159
.p;”* < .
0.01
0.0003 0.0005 0.0007 0.0009 0.001 3
0.001 8 0.0025 0.0034 0.0045 0.0060
0.0080 0.01 04 0.01 36 0.01 74 0.0222
0.0281 0.0351 0.0436 0.0537 0.0655
0.0793 0.0951 0.1 131 0.1335 0.1562
0.1814 0.2090 0.2389 0.2709 0.3050
0.3409 0.3783 0.4168 0.4562 0.4960
0.5040 0.5438 0.5832 0.621 7 0.6591
0.6950 0.7291 0.761 1 0.7910 0.81 86 ,
CI N/OU5RI /TABF5.XLS/5-94
0.02
0.0003 0.0005 0.0006 0.0009 0.0013
0.001 8 0.0024 0.0033 0.0044 0.0059
0.0078 0.01 02 0.0132 0.01 70 0.021 7
0.0274 0.0344 0.0427 0.0526 0.0643
0.0778 0.0934 0.1112 0.1314 0.1539
0.1788 0.2061 0.2358 0.2676 0.3015
0.3372 0.3745 0.41 29 0.4522 0.4920
0.5080 0.5478 0.5871 0.6255 0.6628
0.6985 0.7324 0.7642 0.7939 0.821 2
0.03
0.0003 0.0004 0.0006 0.0009 0.001 2
0.001 7 0.0023 0.0032 0.0043 0.0057
0.0075 0.0099 0.01 29 0.01 66 0.021 2
0.0268 0.0336 0.041 8 0.0516 0.0630
0.0764 0.091 8 0.1093 0.1292 0.1515
0.1 762 0.2033 0.2327 0.2643 0.2981
0.3336 0.3707 0.4090 0.4483 0.4880 0.51 20 0.551 7 0.591 0 0.6293 0.6664
0.7019 0.7357 0.7673 0.7967 0.8238
0.04
0.0003 O.OOO4 0.0006 0.0008 0.0012
0.001 6 0.0023 0.0031 0.0041 0.0055
0.0073 0.0096 0.01 25 0.01 62 0.0207
0.0262 0.0329 0.0409 0.0505 0.061 8
0.0749 0.0901 0.1075 0.1271 0.1492
0.1736 0.2005 0.2296 0.261 1 0.2946
0.3300 0.3669 0.4052 0.4443 0.4840 0.5160 0.5557 0.5948 0.6331 0.6700
0.7054 0.7389 0.7704 0.7995 0.8264
F-16
0.05
0.0003 0.0004 O.OOO6 0.0008 0.001 1
0.001 6 0.0022 0.0030 0.0040 0.0054
0.0071 0.0094 0.01 22 0.01 58 0.0202
0.0256 0.0322 0.0401 0.0495 0.0606
0.0735 0.0885 0.1056 0.1251 0.1469
0.1711 0.1977 0.2266 0.2578 0.291 2
0.3264 0.3632 0.401 3 0.4404 0.4801
0.5199 0.5596 0.5987 0.6368 0.6736
0.7088 0.7422 0.7734 0.8023 0.8289
0.06
0.0003 0.0004 0.0006 0.0008 0.001 1
0.001 5 0.0021 0.0029 0.0039 0.0052
0.0069 0.0091 0.01 19 0.0154 0.0197
0.0250 0.031 4 0.0392 0.0485 0.0594
0.0721 0.0869 0.1038 0.1230 0.1446
0.1685 0.1949 0.2236 0.2546 0.2877
0.3228 0.3594 0.3974 0.4364 0.4761
0.5239 0.5636 0.6026 0.6406 0.6772
0.7123 0.7454 0.7764 0.8051 0.8315
0.07
0.0003 0.0004 0.0005 0.0008 0.001 1
0.0015 0.0021 0.0028 0.0038 0.0051
0.0068 0.0089 0.01 16 0.0150 0.01 92
0.0244 0.0307 0.0384 0.0475 0.0582
0.0708 0.0853 0.1020 0.1210 0.1423
0.1660 0.1922 0.2206 0.251 4 0.2843
0.31 92 0.3557 0.3936 0.4325 0.4721
0.5279 0.5675 0.6064 0.6443 0.6808
0.7157 0.7486 0.7794 0.8078 0.8340
0.08
0.0003 0.0004 0.0005 0.0007 0.0010
0.0014 0.0020 0.0027 0.0037 0.0049
0.0066 0.0087 0.01 13 0.0146 0.0188
0.0239 0.0301 0.0375 0.0465 0.0571
0.0694 0.0838 0.1003 0.1 190 0.1401
0.1635 0.1894 0.21 77 0.2483 0.281 0
0.31 56 0.3520 0.3897 0.4286 0.4681
0.5319 0.5714 0.61 03 0.6480 0.6844
0.7190 0.751 7 0.7823 0.81 06 0.8365
000941
0.09
0.0002 0.0003 0.0005 0.0007 0.0010
0.001 4 0.001 9 0.0026 0.0036 0.0048
0.0064 0.0084 0.01 10 0.0143 0.01 83
0.0233 0.0294 0.0367 0.0455 0.0559
0.0681 0.0823 0.0985 0.1170 0.1379
0.161 1 0.1867 0.2148 0.2451 0.2776
0.3121 0.3483 0.3859 0.4247 0.4641
05359 0.5753 0.61 41 0.651 7 0.6879
0.7224 0.7549 0.7852 0.81 33 0.8389
- Z 1 .o 1.1 1.2 1.3 1.4
1.5 1.6 1.7 1.8 1.9
2.0 2.1 2.2 2.3 2.4
2.5 2.6 2.7 2.8 2.9
3.0 3.1 3.2 3.3 3.4
-
-
-
-
-
-
0.04
0.8508 0.8729 0.8925 0.9099 0.9251
0.9382 0.9495 0.9591 0.9671 0.9738
0.9793 0.9838 0.9875 0.9904 0.9927
0.9945 0.9959 0.9969 0.9977 0.9984
0.9988 0.9992 0.9994 0.9996 0.9997
0 0.8413 0.8643 0.8849 0.9032 0.91 92
0.9332 0.9452 0.9554 0.9641 0.971 3
0.9772 0.9821 0.9861 0.9893 0.991 8
0.9938 0.9953 0.9965 0.9974 0.9981
0.9987 0.9990 0.9993 0.9995 0.9997
0.05
0.8531 0.8749 0.8944 0.9115 0.9265
0.9394 0.9505 0.9599 0.9678 0.9744
0.9798 0.9842 0.9878 0.9906 0.9929
0.9946 0.9960 0.9970 0.9978 0.9984
0.9989 0.9992 0.9994 0.9996 0.9997
pr 0.8438 0.8665 0.8869 0.9049
~ 0.9207
0.9345 0.9463 0.9564
0.9778 0.9826
0.9896 0.9920
0.9955 0.9966 0.9975 0.9982
0.9987 0.9991 0.9993 0.9995 0.9997
0.06
0.8554 0.8770 0.8962 0.91 31 0.9279
0.9406 0.9515 0.9608 0.9686 0.9750
0.9803 0.9846 0.9881 0.9909 0.9931
0.9948 0.9961 0.9971 0.9979 0.9985
0.9989 0.9992 0.9994 0.9996 0.9997
FEMP Background Study May 1994
0.07
0.8577 0.8790 0.8980 0.91 47 0.9292
0.9418 0.9525 0.9616 0.9693 0.9756
0.9808 0.9850 0.9884 0.9911 0.9932
0.9949 0.9962 0.9972 0.9979 0.9985
0.9989 0.9992 0.9995 0.9996 0.9997
Stands
0.02
0.8461 0.8686 0.8888 0.9066 0.9222
0.9357 0.9474 0.9573 0.9656 0.9726
0.9783 0.9830 0.9868 0.9898 0.9922
0.9941 0.9956 0.9967 0.9976 0.9982
0.9987 0.9991 0.9994 0.9995 0.9997
Tablc rd Nom
0.03
0.8485 0.8708 0.8907 0.9082 0.9236
0.9370 0.9484 0.9582 0.9664 0.9732
0.9788 0.9834 0.9871 0.9901 0.9925
0.9943 0.9957 0.9968 0.9977 0.9983
0.9988 0.9991 0.- 0.9996 0.9997
- -
-
-
-
-
-
F-5 (Continued) I Curve for a 2 Distribution
0.08
0.8599 0.881 0 0.0997 0.9162 0.9306
0.9429 0.9535 0.9625 0.9699 0.9761
0.981 2 0.9854 0.9887 0.9913 0.9934
0.9951 0.9963 0.9973 0.9980 0.9986
0.9990 0.9993 0.9995 0.9996 0.9997
0.09
0.8621 0.8830 0.901 5 0.9177 0.931 9
0.9441 0.9545 0.9633 0.9706 0.9767
0.981 7 0.9857 0.9890 0.991 6 0.9936
0.9952 0.9964 0.9974 0.9981 0.9986
0.9990 0.9993 0.9995 0.9997 0.9998
Source: Devore, J.L., 1982. "Probability & Statistics for Engineering and the Sciences", Table A-3, Brooks/Cole Publishing Company, Monterey, CA.
. _
ClN/OU5Rl/rABFSlX~S/5-94
000942
F-17
FEMP Background Study May1994
TABLE F-6
PERCENTAGE POINTS OF THE WI TEST FOR n > 50
~ ~~
n 0.01 0.05
51 53 55 57 59 61 63 65 67 69 71 73 75 77 79
81 83 85 87 89 91 93 95 97 99
0.935 0.938 0.940 0.944 0.945 0.947 0.947 0.948 0.950 0.95 1
0.953 0.956 0.956 0.957 0.957
0.958 0.960 0.96 1 0.96 1 0.961 0.962 0.963 0.965 0.965 0.967
0.954 0.957 0.958 0.96 1 0.962
0.963 0.964 0.965 0.966 0.966 0.967 0.968 0.969 0.969 0.970
0.970 0.97 1 0.972 0.972 0.972 0.973 0.973 0.974 0.975 0.976
~~~ ~~
Source: Table A-3, U.S. EPA 1992.
CIN/OUSRI/WP/419195/TableF-7/5-94
. . ‘ I . . - . ”* I
F-18
000943
FEW Background Study May 1994
To test the null hypothesis that the population has a lognormal distribution, transform the observed data to yi, y2...yn where yi = In xi. Repeat steps 1 through 10 as described above.
Example:
To illustrate the application of the Shapiro-Francia test, the data in Table F-7 is used.
1. Order the n data points in ascending order (smallest to largest) as in Column 2.
2. Compute (i/(n+ 1)) where n is the total number of samples (n = 83) which is presented in Column 3.
3. Compute the normal quantile (q) corresponding to Column 3 probabilities. The normal quantiles are presented in Column 4.
4. Compute the square of the normal quantiles which are presented in Column 5.
5. Compute the multiplication of the normal quantiles (Column 4) times the corresponding data point (Column 2). These results are presented in Column 6.
6. Compute the arithmetic standard deviation.
s = 0.019 (See arithmetic standard deviation example for the procedure for calculation of this value; note that the data sets are not the same.)
7. Compute the denominator of the Shapiro-Francia Test.
n
i = l (n-1) s2 rnf = 82 (0.019)* (75.611)
= 2.238
F-19 000944
FEMP Background Study May 1994 0
8. Compute the numerator of the Shapiro-Francia Test.
9. Compute W*. 0.55 1 w' = - 2.238
= 0.246
10. Reject H, at the 0.05 significance level because W I = 0.246 is less than Weritifal of 0.985, the quantile given in Table F-6, and concluded that the data were not drawn from a population with an underlying normal distribution.
The Shapiro-Francia Test was repeated after the transformation of yi = In W I test 7 0.659. Because the calculated value is less than the critical value (Wcritica, = 0.985), it is concluded that the data set was not drawn from a population having an underlying lognormal distribution.
resulted in
When the W I test fails to reject the null hypotheses for both the normal and lognormal distributions, the W test value which exceeds the W I critifal value the most is selected as the distribution for the data set. When the W test rejects the null hypotheses for both the normal and lognormal distribution but one is close (W I > 0.95 W Icritical) to the test value, then this distribution is (normal or lognormal) selected. If both the normal and lognormal W test are not close to W I critical, then the distribution is considered to be undefined and nonparametric statistical analyses will be used to describe the data set.
F-20
Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
-
Table F-7 Example Data Set Number 2
- xi
0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001
-
CI N/OU5R1/419195/rABF6.X~S/5-~
(i/(n+ 1)) 0.01 2 0.024 0.036 0.048 0.060 0.071 0.083 0.095 0.107 0.119 0.131 0.143 0.155 0.167 0.179 0.190 0.202 0.214 0.226 0.238 0.250 0.262 0.274 0.286 0.298 0.31 0 0.321 0.333 0.345 0.357 0.369 0.381 0.393 0.405 0.41 7 0.429 0.440 0.452 0.464 0.476 0.488 0.500 0.512 0.524 0.536 0.548 0.560 0.571 0.583 0.595
Normal quantile, mi
-2.260 -1 981 -1.803 -1.668 -1.559 -1.465 -1 -383 -1.309 -1.242 -1.180 -1.122 -1 .om -1.016 -0.967 -0.921 -0.876 -0.833 -0.792 -0.751 -0.71 2 -0.674 -0.637 -0.601 -0.566 -0.531 -0.497 -0.464 -0.431 -0.398 -0.366 -0.334 -0.303 -0.272 -0.241 -0.210 -0.180 -0.150 -0.120 -0.090 -0.060 -0.030 0.000 0.030 0.060 0.090 0.120 0.150 0.180 0.21 0 0.241
F-21
mi-2 5.108 3.923 3.250 2.784 2.430 2.147 1.913 1.714 1.542 1.392 1.259 1.140 1.033 0.936 0.848 0.768 0.694 0.627 0.565 0.508 0.455 0.406 0.362 0.320 0.282 0.247 0.215 0.186 0.159 0.134 0.1 12 0.092 0.074 0.058 0.044 0.032 0.022 0.014 0.008 0.004 0.001 0.000 0.001 0.004 0.008 0.014 0.022 0.032 0.044 0.058
FEMP Background Study May 1994
mi*xi -0.002 -0.002 -0.002 -0.002 -0.002 -0.001 -0.001 -0.001 -0.001 -0.001 4.001 -0.001 4.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
000946
FEMP Background Study May 1994
- Rank 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83
Total
-
- -
xi 0.001 0.001 0.001 0.001 0.001 4 0.0015 0.001 5 0.0016 0.002 0.002 0.0025 0.0025 0.0025 0.0025 0.0026 0.003 0.004 0.004 0.004 0.0044 0.005 0.005 0.005 0.005 0.006 0.006 0.009 0.01 1 0.028 0.029 0.06 0.08 0.14
-
-
Table F-7 (Continued) Example Data Set Number 2
(i/(n+ 1)) 0.607 0.61 9 0.631 0.643 0.655 0.667 0.679 0.690 0.702 0.71 4 0.726 0.738 0.750 0.762 0.774 0.786 0.798 0.810 0.821 0.833 0.845 0.857 0.869 0.881 0.893 0.905 0.91 7 0.929 0.940 0.952 0.964 0.976 0.988
Normal quantile, mi
0.272 0.303 0.334 0.366 0.398 0.431 0.464 0.497 0.531 0.566 0.601 0.637 0.674 0.712 0.751 0.792 0.833 0.876 0.921 0.967 1.016 1.068 1.122 1.180 1.242 1.309 1.383 1.465 1.559 1.668 1 .a03 1.981 2.260
F-22
____
mi-2 0.074 0.092 0.112 0.134 0.159 0.186 0.215 0.247 0.282 0.320 0.362 0.406 0.455 0.508 0.565 0.627 0.694 0.768 0.848 0.936 1.033 1.140 1.259 1.392 1.542 1.714 1.913 2.147 2.430 2.784 3.250 3.923 5.108 75.61 1
ml*xi 0.000 0.000 0.000 0.000 0.001 0.001 0.001 0.001 0.001 0.001 0.002 0.002 0.002 0.002 0.002 0.002 0.003 0.004 0.004 0.004 0.005 0.005 0.006 0.006 0.007 0.008 0.012 0.016 0.044 0.048 0.108 0.158 0.31 6 0.742
000947
i
FEMP Background Study May 1994
Rosner's Test for Many Outliers
To use Rosner's Test (Gilbert 1987, Equations 15.1 to 15.3) it is necessary to specify an upper limit of the number of potential outliers present. This analysis was performed for up to ten outliers. Rosner's Test requires the calculation of a test statistic Ri+l using the following equation:
where:
Ri+l = test statistic for deciding whether the i + 1 most extreme values in the complete data set are statistical outliers
i = 0 for the first suspected outlier
i = 1 for the second suspected outlier
i = 9 for the tenth suspected outlier
= sample arithmetic mean of the remaining data set after the i most extreme observations have been deleted
F-23
000948
FEW Background Study May 1994
s(') = sample standard deviation of the remaining data set after the i most extreme observations have been deleted
and 5 = jth observation in the data set
n = total number of observations in the data set
A suspected extreme value is determined to be an outlier if the calculated value of Ri+l exceeds the critical value Ai+l for a sample of size n (Table F-8).
In applying Rosner's Test when there is only one suspected outlier, i = 0 and
and x(O) = suspected outlier
2') = sample arithmetic mean of the n observations including the suspected outlier
s(O) = sample arithmetic standard deviation of the n observations including the suspected outlier
F-24
564% FEMP Background Study
May 1994
Table F-6 Approximate Critical Values Lamda (i+ 1) for Rosnets Generalized
ESD Many-Outlier Procedure for alpha = 0.05
n 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 60 70 80 90 100 150 200 250 300 350 400 450 500 750 1000 2000 3000 4000 5000
1 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.10 3.1 1 3.12 3.13 3.20 3.26 3.31 3.35 3.38 3.52 3.61 3.67 3.72 3.77 3.80 3.84 3.86 3.95 4.02 4.20 4.29 4.36 4.41
2 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.10 3.1 1 3.12 3.19 3.25 3.30 3.34 3.38 3.51 3.60 3.67 3.72 3.77 3.80 3.84 3.86 3.95 4.02 4.20 4.29 4.36 4.41
Source: Table A-1 6, Gilbert 1987.
3 2.78 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.10 3.1 1 3.19 3.25 3.30 3.34 3.38 3.51 3.60 3.67 3.72 3.77 3.80 3.84 3.86 3.95 4.02 4.20 4.29 4.36 4.41
4 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.10 3.18 3.24 3.29 3.34 3.37 3.51 3.60 3.67 3.72 3.77 3.80 3.84 3.86 3.95 4.02 4.20 4.29 4.36 4.41
5 2.73 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.17 3.24 3.29 3.33 3.37 3.5i 3.60 3.67 3.72 3.76 3.80 3.83 3.86 3.95 4.02 4.20 4.29 4.36 4.41
10 2.59 2.62 2.65
, 2.68 2.71 2.73 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.14 3.21 3.26 3.31 3.35 3.50 3.59 3.66 3.71 3.76 3.80 3.83 3.86 3.95 4.02 4.20 4.29 4.36 4.41
CIN/OU5RI/419195/rABF8.XLS/5-94
1 *.<,. I . - .
F-25
FEW Background Study May 1994
When Rosner's Test is applied in a situation where there are two suspectd outliers, i= 1 and
and
x(') = the second suspected outlier
3') = the sample arithmetic mean after the first suspected outlier x(O) has been deleted from the data set
dl) = the sample arithmetic standard deviation after the first suspected outlier x(O) has been deleted from the data set
Example:
Table F-9 presents example data set number 3 which is used for this example. The highest and lowest posted concentrations of 4850 pg/L and 189 pg/L, respectively, are considered to be potential outliers. To begin with, Rosner's Test is applied to determine whether or not the value of 4850 pg/L is a statistical outlier. In this case, i = 0 and
The da I are best described by a lognormal distribution and as such
where:
Y'O) = x(o)
= In (4850)
= 8.487
F-26
. . . J
000959
FEMP B a c k g r o R g t
Chemical Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese
Table F-9 Example Data Set Number 3
Validated result, ug/L 189 301 351 370 386 422 437 451 456 481 488 521 534 535 543 581 61 5 61 9 747 766 785 840 941 1050 1070 1090 1150 1460 1500 4850
Validation qualifier
J
J J
J
J
Concentration used in stat
Normal 189 30 1 35 1 370 386 422 437 451 456 481 488 521 534 535 543 581 61 5 61 9 747 766 785 840 941 1050 1070 1090 1150 1460 1500 4850
.its (a) Lognormal
5.242 5.707 5.861 5.914 5.956 6.045 6.080 6.1 11 6.122 6.176 6.190 6.256 6.280 6.282 6.297 6.365 6.422 6.428 6.61 6 6.641 6.666 6.733 6.847 6.957 6.975 6.994 7.048 7.286 7.31 3 8.487
(a) When the validation qualifier contains a 'U", then one-half of the concentration is used in the statistical calculations.
F-27
FEMP Background Study May 1994
To) = sample arithmetic mean of In x using all 30 observations in the data set
= 6.477
s(O) = sample arithmetic standard deviation of In x using all 30 observations in the data set
= 0.6098
Thus
18.487 - 6.477) 0.6098
R, =
= 3.296
Because R, = 3.296 is greater than the critical value of Ai+, = 2.91 (Table F-8) for a significance level of 0.05, Rosner's Test indicates that the observed value of 4850 pg/L is a statistical outlier.
Next Rosner's Test is used to determine whether or not there is sufficient evidence to conclude that the minimum value of 189 pg/L is a statistical outlier.
After deleting the maximum value of 4850 pg/L from the data set, calculate
where:
y") = In (189)
= 5.242
yl) = 6.407
s(l) = 0.4889
i 5-64.4
FEMF' Background Study May 1994
Then
15.242 - 6.4071 R, = 0.489
= 2.38
Because R, = 2.38 is less than the critical value of 1, = 2.89 (Table F-8) for a significance level of 0.05, Rosner's Test indicates that there is insufficient evidence to reject the observed value of 189 pg/L. The value is not a statistical outlier. This test is repeated on the high and low ends until there is insufficient evidence to reject the null hypothesis or 10 outliers are identified. This test identifies outliers but professional judgment was used to determine whether the value should be removed from the data set.
F-29 .
000954
EMF' Background Study May 1994
Data Averaging
Data averaging was conducted when two or more samples were collected at the same sampling location on the same day (Le., duplicates, triplicates, etc.). Multiple samples collected on a particular day and location were averaged to obtain one sample per day at the sampling location. Data averaging was conducted to avoid statistical bias which would have resulted from favoring one day's multiple sampling over one sample on a particular day. The data was separated into data groups by the following: media (Le., Glacial Overburden, Great Miami Aquifer, Great Miami River, and Paddys Run), constituent type (Le., radiological, inorganic, and organic constituents), and filter type (Le., filtered and unfiltered). The following steps were repeated for each of the above data groups:
1. Nondetect values were assigned a value of one-half the detection limit.
2. The data were sorted by three key parameters: 1) chemical name, 2) sample loca- tion, and 3) sampling date.
3. A record-by-record comparison was performed to identify records for which all three sort parameters were the same. This step identified multiple samples collected on a particular day for a specific constituent.
4. A sample arithmetic mean was calculated on the records with matching key parame- ters.
5. If all of the values being averaged were nondetects, the average was also a nondetect. If one or more of the values being averaged were detect values, then the resulting average was also considered to be a detect value.
6. A new line was added to the database for the averages of the matching records.
7. The individual records that were combined to create the average line were removed from the statistical data set.
Example:
The example data set is presented in Table F-10. To simplify this example, only one constituent was selected. Seven ammonia samples were collected from 1988 to 1993. The following steps were performed to calculate the averaged values that were used in the statistical analysis:
1. Nondetect values were assigned a value of one-half the detection limit. This occurred for five of the seven values (i.e., c 1 equals 0.05).
CrN/OUS~/wp/4191%/AppENDIxF/s-W F-30 i A ' , > L .. , 000955
FEMP Background Study May 1994
05/20/93 06/23/93 06/23/93 08/29/88 04/03/89 05/20/93 05/20/93 05/20/93 06/23/93 06/23/93 05/20/93 05/20/93 06/23/93
Table F-10 Example Data Set Number 4
120072-2 12041 6 12041 4 1092 1178 120064-2 120068-1 120072-2 12041 6 120414 120068-1 120072-2 12041 6
Well No. w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1
0.1 1 0.1 . 0.1 0.1 0.1
Sample
N UJ N UJ D UJ T U N
05/20/93 120072-2 06/23/93 I 12041 6
0.002 0.0039 0.0032 0.0025
U N D T
J N
04/03/89 05/20/93 05/20/93 06/23/93
08/29/88 04/03/89 05/20/93 05/20/93 06/23/93
1178 120068-1 120072-2 12041 6
1092 1178 120068-1 120072-2 12041 6
0.1 0.0493 0.0884 0.0893 0.0906
05/20/93 120072-2 06/23/93 I 12041 6
N N D T N
0.002 0.0098 0.005 0.005 0.005 0.005
77 70.1 61.2 76.5 77.1 72.3 68.4
Lab qualifier
U N N
U D U T U N U D
N N N D T N
- . D
U U U U U U U
uw uw U U B B
BW U
B B B B U U U U
U
U U U U
08/29/88 04/03/89 05/20/93 05/20/93 06/23/93
Constituent Alkalinitv as CaC03
1092 1178 120068-1 120072-2 12041 6
Alkalinitjl as Alkalinity as CaC03 Alkalinity as CaC03 Aluminum Aluminum Aluminum Aluminum Ammonia Ammonia Ammonia Ammonia Ammonia Ammonia Ammonia Antimony Antimony Antimony Antimony Arsenic Arsenic Arsenic Arsenic Arsenic Arsenic Barium Barium Barium Barium Barium Barium Barium Beryllium Beryllium Beryllium Beryllium Cadmium Cadmium Cadmium Cadmium Cadmium Cadmium Cadmium Calcium Calcium Calcium Calcium Calcium Calcium Calcium
F-31
233 I:, 230 I I D 1.27 I I D
2.14 1-33 I 1 I:, 1.64 I I D 0.1 I J I N
0.1 I U I D 0.005 I U I D
0.005 I:, 0.005 1 UJ I D 0.005 I U I N
0.002 I U I D 0.089 I I N
0.0847 1 I D 0.002 I U I D
0.002 O*Oo2 I u I:, 0.002 I U I D 0.006 I I N
- 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 3 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 4 3 3 3 3 3 3 4 3 3 3 3 3
-
-
-
-
-
-
-
-
-
000956
2.
3.
4.
5.
6.
7 .
FEW Background Study May 1994
The data were sorted by three key parameters: 1) chemical name (ammonia), 2) sample location (W-1), and 3) sampling date. There only was one sampling location for W-1 and W-5; however, multiple sampling locations (wells) were sampled for the glacial overburden and Great Miami Aquifer (5 and 24, respectively).
A record-by-record comparison identified the following matches: three samples collected on May 20, 1993 and two samples collected on June 23, 1993. Individual records were identified on August 29, 1988 and April 3, 1989.
All samples collected on May 20 and June 23, 1993, had nondetect values of ~ 0 . 1 (0.05 for statistics); therefore, the averaged value for these two days was a nondetect value of e 0.1.
The averaged values are both nondetects, since all of the data used to calculate the average value were nondetects.
New lines were added to the database for these two averaged records from the matching key parameters.
The five lines that were combined to create the averaged records were removed from the statistical database.
F-32
00095'7
F" Background Study May 1994
Sample Arithmetic Mean - Normal Distribution
The sample arithmetic mean for a normal distribution is given by:
Example:
Using example data set number 1 presented in Table F-4, the sample arithmetic mean is calculated as follows:
- X =
= 0.045
C0.017 + 0.0245 + .... + 0.066 + 0.073]/36
000958
FEMP Background Study May 1994
Sample Arithmetic Standard Deviation - Normal Distribution
The sample arithmetic standard deviation for a population based on a sample is given by:
Example:
n
i=l c (3 - 3 2
- - n-1 4
2 n 4 2 - [ E 31 In
i=l i=l n-1
Using example data set number 1 presented in Table F-4, the sample arithmetic standard deviation for a normal distribution is calculated as follows where n = 36 and:
n
i=l xi2 = (0.017)2 + (0.0245)2 + ....+ (0.066)2 + (0.073)2
= 0.081
[ $ 3[ = [0.017 +0.0245 + .... + 0.066 + 0.07312 i=l
= 2.670
= 1 0.081 - 2.670136 36 - 1
= 0.014
FEMP Background Study May 1994
Estimated Coefficient of Variation - Normal Distribution
The estimated coefficient of variation for a normal distribution is calculated by the following:
cv = sb
where: - x = arithmetic mean s = sample standard deviation CV = estimated coefficient of variation
Example:
Using example data set number 1 presented in Table F-4, the estimated coefficient of variation for a normal distribution is calculated as follows: e - x = 0.045
s = 0.014
CV = 0.014/0.045 = 0.31
FEMP Background Study May 1994
Estimated Mean of a Lognormal Distribution
The estimated mean of a lognormal distribution (Gilbert 1987, Equation 13.7) can be calculated by using the sample mean (y) and the sample standard deviation (s,.) of the log-transformed data. The formula is as follows:
Example:
Using the data in Table F-4, the estimated mean of a lognormal distribution is calculated as follows where n = 36 and:
n
[ln(0.017) + In(0.0245) +.....+ ln(0.066) WX,) - i = l
Y = 36 n
- -113.201 - 36
= -3.144
+ =
n
n - 1 n - 1
FEW Background Study May 1994
F-37
n C p(%)p = h1(0.017)~ + ln(O.O245)* +....+ 1n(0.066)2 + ln(0.073)2
i = l
= 360.063
n
i = l p(+)] = ln(0.017) + ln(0.0245) +....+ ln(0.066) + ln(0.073)
= -113.201
360.063 - (-113.201)2/36 36-1
= 0.342
1 ( 0.342)2 2
= 0.046
FEMP Background Study May 1994
Estimated Standard Deviation of a Lognormal Distribution
The estimated standard deviation of a lognormal distribution (Gilbert 1987, Equation 13.8) can be calculated by using the estimate for the mean of the lognormal distribution (ii) and the sample standard deviation (3) of the log-transformed data. The formula is as follows:
Example:
Data in Table F-4 were used to calculate the estimated standard deviation of a lognormal distribution. The parameters used in this example were presented in the example for estimating the mean of a lognormal distribution where 0 = 0.046 and 3 = 0.342. Therefore, the estimated standard deviation of a lognormal distribution is calculated as follows:
6 = (k0.046)2 [exp (0.342)2 - 11
= 0.0162
- .
963
5644 FEMP Background Study
May 1994
Sample Median - Nonparametric Technique
The true median of an underlying distribution (Gilbert 1989, Equations 13.15 and 13.16) is that value above which and below which half of the distribution lies. The median of any distriubtion, no matter what it's shape, can be estimated by the following:
Sample median = x [ ( ~ + ~ ) / ~ ~ if n is odd
where:
xpl = i* data value in the ordered data set
Example:
Using example data set number 1 presented in Table F-4, the sample median is calculated as follows where n = 36 and:
- - '18 + '19
2
= (0.045 + 0.045)/2
= 0.045
If n were 35 rather than 36, then the median would be at XI8 = 0.045.
000964 F-39
FEW Background Study May 1994
Upper One-sided 95% Confidence Limit - Normal Distribution
The mean el) for a sample of size n is referred to as a point estimate of the true but unknown population mean (p) . If a second sample of size n is drawn from the sample population, the sample mean &) will most likely not be equal toil. In fact if the sampling process is replicated many times, the sample means themselves will have a distribution. Further, the distribution of means of samples of size n will tend toward a normal distribution, if n is sufficiently large.
The 100 (1-a) Upper Confidence Limit (Gilbert 1987, Equation 11.6) of the population mean ( p ) can also be calculated. When a = 0.05, the upper one-sided 95 percent confidence limit is:
S 95% UCL, = x' + to*ss,-, - J;;
where:
toass a-l = value from the "t" distribution in Table F-11.
It should be noted that the 95 percent confidence limit for a second sample of size n drawn from the same population will most likely not be the same as that for the first sample. In theory if a limit estimate is calculated for the means of a very large set of samples of size n, the true population mean will be less than the limit 95 percent of these limits. If the number of degrees of freedom is not listed in Table F-11, then linear interpolation was performed to obtain the t-value.
F-40 000965
FEMP Backgro
Degrees of freedom
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 40 60 1 20
Infinite
Table F-11 Quantiles of the t Distribution (Values oft Such that loop%
of the Distribution Is Less Than tp)
t at 0.60 0.325 0.289 0.277 0.271 0.267 0.265 0.263 0.262 0.261 0.260 0.260 0.259 0.259 0.258 0.258 0.258 0.257 0.257 0.257 0.257 0.257 0.256 0.256 0.256 0.256 0.256 0.256 0.256 0.256 0.256 0.255 0.254 0.254 0.253
t at 0.70 0.727 0.61 7 0.584 0.569 0.559 0.553 0.549 0.546 0.543 0.542 0.540 0.539 0.538 0.537 0.536 0.535 0.534 0.534 0.533 0.533 0.532 0.532 0.532 0.531 0.531 0.531 0.531 0.530 0.530 0.530 0.529 0.527 0.526 0.524
jource: Table A-2, Gilbert 1987.
. , , : , > ” , ., . , -. ‘.I
ClN/OU5RI/WP/419195/rABFll .XLS/5-94
t at 0.80 1.376 1.061 0.978 0.941 0.920 0.906 0.896 0.889 0.883 0.879 0.876 0.873 0.870 0.868 0.866 0.865 0.863 0.862 0.861 0.860 0.859 0.858 0.858 0.857 0.856 0.856 0.855 0.855 0.854 0.854 0.851 0.848 0.845 0.842
t at 0.90 3.078 1.886 1.638 1.533 1.476 1.440 1.415 1.397 1.383 1.372 1.363 1.356 1.350 1.345 1.341 1.337 1.333 1.330 1.328 1.325 1.323 1.321 1.319 1.318 1.316 1.315 1.314 1.313 1.31 1 1.310 1.303 1.296 1.289 1.282
t at 0.95 6.31 4 2.920 2.353 2.132 2.015 1.943 1.895 1.860 1.833 1.812 1.796 1.782 1.771 1.761 1.753 1.746 1.740 1.734 1.729 1.725 1.721 1.71 7 1.714 1.71 1 1.708 1.706 1.703 1.701 1.699 1.697 1.684 1.671 1.658 1.645
t at 0.975 12.706 4.303 3.182 2.776 2.571 2.447 2.365 2.306 2.262 2.228 2.201 2.179 2.160 2.145 2.131 2.120 2.1 10 2.101 2.093 2.086 2.080 2.074 2.069 2.064 2.060 2.056 2.052 2.048 2.045 2.042 2.021 2.000 1.980 1.960
t at 0.990 31.821 6.965 4.541 3.747 3.365 3.143 2.998 2.896 2.821 2.764 2.71 8 2.681 2.650 2.624 2.602 2.583 2.567 2.552 2.539 2.528 2.51 8 2.508 2.500 2.492 2.485 2.479 2.473 2.467 2.462 2.457 2.423 2.390 2.358 2.326
t at 0.995 63.656 9.925 5.841 4.604 4.032 3.707 3.499 3.355 3.250 3.169 3.106 3.055 3.01 2 2.977 2.947 2.921 2.898 2.87% 2.861 2.845 2.831 2.819 2.807 2.797 2.787 2.779 2.771 2.763 2.756 2.750 2.704 2.660 2.61 7 2.576
000966 F-41
FEW Background Study May 1994
Example:
Using example data set number 1 in Table F-4, the upper one-sided 95 percent confidence limit for a normal distribution was calculated as follows:
n = 36
X = 0.045 S = 0.014
-
h.Es = 1.6905
0.014 95% UCL, = 0.045 + (1.6905) - m = 0.049
FEMP Background Study May 1994
Upper One-sided 95% Confidence Limit - hgnormal Distribution
The procedure for calculating the upper 95% confidence limit for the lognormal distribution (Gilbert 1987, Equation 13.13) is given by:
2 sy H0.W + 0.5 sy + -
4 x - 95% UCL, = exp
where:
y = l n x - y = arithmetic mean of y
sY = standard deviation of y
n = number of data points
H,,, = value from Table F-12 for sample of size n
Example:
The data presented in Table F-4 were used for this example. The mean of the log transformed data (y) equals -3.144 and the standard deviation of the log transformed data equals 0.342. These were previously calculated in the section on calculating the estimated mean of a lognormal distribution. Tables for determining H values are presented individually by number of data points for various significance levels. A summary of these tables for a 0.05 significance level (Ho.95) is presented in Table F-12. When the number of data points was not listed in this Table F-12 then linear interpolation between columns was performed. In addition, if sy did not match the sy in the first column of Table F-12, then linear interpolation between column values was performed. The upper 95% one-sided confidence limit for the lognormal distribution is calculated as follows:
where: n = 36 y = -3.144 (see estimated mean of lognormal distribution example) s = 0.342
= 1.807 (linear interpolation between sy values)
F-43
FEMP Background Study May 1994 i
I ! - I I I I I
000969 CIN~OlJ5dl/'WP/419195/TABF12.XLS/5-94 F-44
FEMP Backgr &I%!# May 1994
a
6 c 0 P E
n Q c P - 9 .- Y
I 0 r
F45
FEW Background Study May 1994 .
(0.342) (1.807) 95% UCL, = exp -3.144 + 0.5(0.342)* + I J%=r = exp (-2.981)
= 0.051
Thus the upper one-sided 95% confidence limit for the lognormal distribution is 0.051.
, cIN/OU5~/wp/419195/AppENDIxp/s-94 F-46 ,.. . . *
000971
5.644.
FEW Background Study May 1994
Upper One-sided 95% Confidence Limit - Nonparametric Technique
The upper 95% one-sided confidence limit for an undefined distribution is based on a noparametric technique (Gilbert 1987, Equation 13.2). It is simply the upper 95% confidence limit on the median of the data set. The following equations are used to calculate the upper one-sided 95% confidence limit for an undefined distribution:
where:
n = number of data points
&.% = upper 95% limit from a standard normal curve for a Z distribution [Table F-5 at 0.95 (&.,) = 1.6451
a U = rank in an ascending order data set that corresponds to the one-sided
95% confidence limit on the median
f(U) = U rounded up to an integer (e.g., 24.2 + 25)
Example:
Using the data presented in Table F-4, the upper one-sided 95% confidence limit on the median is calculated by the following steps.
1. Order the data in ascending order
2. Detennine the number of data points (36)
3. Obtain the &.9s from Table F-5 (1.645)
F-47
FEMP Background Study May 1994
4. Calculate U
36+1 + 1.645 U =
= 23.435
5. Round up to an integer (24)
6. Determine the "Uth value in the ascending order data set value at the 24th rank = (0.514)
The 95% UCL, for this data set is 0.514.
F-48
5644
FEW Background Study May 1994
95* Percentile - Normal Distribution
The 95* Percentile (or Quantile) for a normal distribution are used to determine maximum background concentrations from data sets that are normally distributed. The 95* percentile for a normal distribution (Gilber 1987, Equation 11.1) is calculated based on the following equation:
where: - X = sample arithmetic mean
S = sample arithmetic standard deviation
= upper 95% limit from a standard normal curve for a 2 distribution [Table F-5 at 0.95 (Gags) = 1.6451
Example:
Using example data set number 1 in Table F-4, the 9Sh percentile for a normal distribution was calculated by the following:
- X = 0.045
S = 0.014
9Sth Percentile, = 0.045 + (1.645) (0.014)
= 0.068
F-49 000974
FEMP Background Study May 1994
95* Percentile - Lognormal Distribution
The 9Sth Percentile (or Quantile) for a lognormal distribution (Gilbert 1987, Equation 1
13.24) is calculated based on the following equation.
where: - y = sample arithmetic mean of y
= sample arithmetic standard deviation of y sY
Z,,, = upper 95% limit from a standard normal curve for a Z distribution [Table F-5 at 0.95 (Z,,,) = 1.6451
Example:
Using example data set number 1 in Table F-4, the 9Sh Percentile for a lognormal distribution was calculated by the following:
- y = -3.144
= 0.342 sY
&*, = 1.645
9Sh PercentiZq = exp [-3.144 + (1.645) (0.342)]
= exp (-2.581)
= 0.0757
. .,. ' '
cIN/OUS~/wp/4l9195/AppENDDLF/s-W F-50
FEW Background Study May 1994
9S” Percentile - Nonparametric Technique
The 95* for an undefined distribution is based on a nonparametric technique. It is calculated by the following equations:
Q = 0.95 n
9Sth Percentile, = x[f(o)l
where:
n = number of data points
Q = rank in an ascending order data set that corresponds to the one-sided 0.95 quantile based on a nonparametric technique
f(Q) = Q rounded up to an integer (e.g., 14.1 + 15)
Example:
Using the example data set presented in Table F-4, the 9Sth percentile based on a nonparametric technique is calculated by the following steps:
1. Order the data in ascending order
2. Determine the number of data point (36)
3. Calculate Q
Q = (0.95) (36)
= 34.2
4. Round Q up to an integer (35)
5. Determine the “Qth value in the ascending order data set (value at 3Sth rank = 0.066)
The 9Sth Percentile, for this data set is 0.066.
F-5 1
.._ :. . . . .5644 . I.
FEW Background Study May 1994
F-Test
The F-test is conducted to test whether there is no difference between two population variances from a combined data set that is normally distributed. By conducting this test on the natural logarithms of each data value, the F-test was used to determine whether there is no difference between population variances from a combined data set that is lognormally distributed. An alpha of 0.05 was selected for this test. The null hypothesis to be tested is:
H,: The populations have equivalent variances (0: = 0:)
versus
The alternative hypothesis Rejection Region for a Level 0.05 Test
The following is the procedure to determine whether the two groups have variances which are statistically significantly different.
1. Calculate the sample variances of the two groups (si and si).
2. Calculate the test statistic.
3. Determine the two Fcritica, values. These critical values are FOeOu, n1 1, n2 1, which is obtained from Table F-13, while FOaWs, n1 1, n2 is obtained from 1/Fo.a, n2 - 1, "1- 1.
4. Compare Ftat versus the Fcritica, values.
Based on the rejection table listed above, when Fta, is between the two Fcritica, values, there is insufficient information to conclude that the sample variances are from two different populations. Some statistical packages provide p-values (significance levels) based on the F-test statistic (Ftat). When using these statistical packages, a p-value (significance level) of 0.05 was used. When the p-value was greater than or equal to 0.05, then the null hypothesis (H,) was not rejected. When the p-value was less than 0.05, then the null hypothesis was rejected or the alternative hypothesis was accepted.
cIN/OUS~/wp/sl91%/AppENDIX.F/s-94 F-52 1 ~ :'. .:
5644 FEMP Background Study.
May 1994
.. . CIN/OU5RI/WP/419195/rABFl3.XLS/5-94 F-53
5644 FEMP Background Study
May 1994
Example:
The log transformed data column from Table F-14 was used to conduct the F-test. The procedure for determining whether the two populations have the same sample variances was calculated as follows:
1. Calculate the sample variances.
s2 = - 1 c (3 -a2 n-1 i=l
s: = 1.415
si = 1.853
2. Calculate the test statistic. 2 2
Ftcst = S l b 2
= 1.41511.853 = 0.764
3. Determine the two Fcritica, values.
F0.Ms,9.14 = 3'21
F0.975,9,14 = 1 / F , . ~ , ~ 4 , 9
= 113.80 = 0.263
No. of data
, points 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Averaae
FEk
Normal 0.05 0.05 0.05 0.05 0.05 0.275 0.275 0.476 0.735 0.756 0.851
1 1 1 1.5 0.541
Table F-14 Example Data Set Number 5
Log- normal -2.996 -2.996 -2.996 -2.996 -2.996 -1.291 -1.291 -0.742 -0.308 -0.280 -0.161 0.000 0.000 0.000 0.405 -1.243
Total Uranium,
ug/L 0.1 0.5 0.7 1 1.1
1.2325 1.791 2 2
3.852
0.1 0.1 0.1 0.1 0.55 0.55 0.476 0.735 0.756 0.851
1 1 1 1.5
Val idat ion qualifier
UJ J J
J
UJ UJ UJ U U* U* J J J J J J
1
Conc. used in stat
Normal 0.05 0.5 0.7 1
1.1 1.2325 1.791 2 2
3.852
1.423
tics (a) Log-
normal -2.996 -0.693 -0.357 O.OO0 0.095 0.209 0.583 0.693 0.693 1.349
-0.042
PriW
:- 5644 FEMP Background Study
May 1994
(a) When the validation qualifier contains a "u", then one-half of the concentration is used in the statistical calculations.
F-55
a FEW Background Study May 1994
4. Compare Ftat versus the Fcritical values.
F0.975, 9, 14 ' Ftest and F0.025, 9, 14 > F t a
0.263 c 0.764 and 3.21 > 0.764
Since Fta is between the two Fcritical values, the null hypothesis (H,) is not rejected. If F, were not between the Fcritica, values, the null hypothesis would be rejected. Based on these results, there is insufficient evidence to conclude that the sample variances are from two different populations (lognormally distributed).
a
FEMP Background Study May 1994
T-Test
The T-test is conducted to test whether there is no difference between two population means with equal variances from a combined data set that is normally distributed. By conducting this test on the natural logarithms at each data value, the T-test was used to determine whether there is no difference between two population means with equal variances from a combined data set that is lognormally distributed. An alpha of 0.05 was selected for this test. The null hypothesis to be tested is:
Ho: The populations have equal means
versus
The alternative hypothesis Rejection Region for a Level 0.05 Test
The following is the procedure to determine whether the two groups have means which are statistically significantly different.
1. Calculate the sample means of the two groups (yl and y2).
2. Calculate the sample variances of the two groups (s: and sl>.
3. Calculate the estimated pooled standard deviation. r 105
4. Calculate the test statistic.
1
nl n2
,+- I
5. Determine the critical values (Tcritica,) from Table F-11.
6. Compare Ttat versus the Tcritid values.
FEMP Background Study May 1994
Based on the rejection table listed above, when the T,, statistic is between the two critical values, there is insufficient information to conclude that the means are from two different populations. Some statistical packages provide p-values (significance levels) based on the Ttat statistics. When using these statistical packages, a p-value (significance level) of less than 0.05 was used. When the p-value was greater than or equal to 0.05, then the null hypothesis (H,) was not rejected. When the p-value was less than 0.05, then the null hypothesis was rejected or the alternative hypothesis was accepted.
Example:
The log transformed data column from Table F-14 was used to conduct the T-test. The procedure for determining whether the two populations have the same means is as follows:
1. Calculate the sample means. - x1 = -0.042 (from Table F-14) - x2 = -1.243 (from Table F-14)
2. Calculate the sample variances.
sf = 1.415 (from F-test example)
= 1.853 (from F-test example)
3. Calculate the estimated pooled standard deviation.
(10 - 1)(1.415) + (15 - s = [ 10 + 15 - 2
= 1.297
F-58 000983
FEMP Background Study May 1994
4. Calculate the test statistic.
- (( -0.042) - (-1.243)) I Tta -
1 10 15
1.297 J - + -
= 2.268
5. Determine tk. 3tical values from Table F-11.
6. Compare Tt, versus the critical values.
Ttcst L To.ws,u and Ttest I -To.ws,u
and 2.268 > -2.069 - 2.268 > 2.069 - Since T,,, is greater than To.97s,u, the null hypothesis is rejected. Based on these results, there is a statistically significant difference between these two means. Furthermore, these data sets are not from the same lognormal distribution.
0
0 '- , .. F-59
FEMP Background Study May 1994
The Wilcoxon Rank Sum Test
The Wilcoxon Rank Sum test is a procedure which can be used to determine whether two sample groups have equivalent means. This test assumes that the distributions of the two populations are identical in shape (variance), but the distributions need not be symmetric. The Wilcoxon Rank Sum test was used when comparing two populations (Le., FEMP vs. private wells in the glacial overburden), while the Kruskal-Wallis test was used when comparing three or more populations (Le., Dry Fork, Ross, and Shandon tributaries of the Great Miami Aquifer). In general, the Wilcoxon Rank Sum test should be employed whenever the proportion of nondetects is greater than 15 percent but less than 90 percent. However, in order to provide valid results, the Wilcoxon Rank Sum test should not be used unless both data sets contain at least four samples. The following equations present a step-by-step procedure for conducting the Wilcoxon Rank Sum test.
1.
2.
3.
Combine the Group 1 with the Group 2 data and rank the ordered values from 1 to N. Assume there are n Group 1 samples and m Group 2 samples so that N = m + n.
Compute the Wilcoxon statistic W:
1 2
n
i= l W = Ei - - n (n + 1)
where Ei are the ranks of the Group 1 samples. (Large values of the statistic W give evidence that the groups are not from the same populations.)
Compute an approximate Ztat. To find the critical value of W, a normal approxi- mation to its distribution is used. The expected value and standard deviation of W under the null hypothesis (Le., the groups are from the same population) are given by the formulas
1 2
E(W) = - mn; .
An approximate Z,,, for the Wilcoxon Rank Sum test may be calculated by the following equations:
000985
FEMP Background Study May 1994
1 W - E ( W ) - -
SD’ (W) where SD’ = [TN % =
The factor of 34 in the numerator serves as a continuity correction since the discrete distribution of the statistic W is being approximated by the continuous normal distribu- tion. If n,m > 10 and ties are present, an adjustment to the approximate Z,,, must be made:
g f& - 1) j =i
+ 1 - N(N - 1)
and g is the number of tied groups and tj is the number of tied data in the jth group.
4. For a one-tailed 0.05 significance level test for Ho versus the HA (Le., the mea- surements from population l tend to exceed those from population 2), reject H, and accept HA if 2, 2 ZO4). For a one-tailed a significance level test for H, versus the HA that the measurements from population 2 tend to exceed those from population 1, reject H, and accept HA if 2, - < Z&).
Example:
The data for this example are presented in Table F-15.
1. Combine the FEMP and private Total Uranium data and rank the ordered values from 1 to 25 as shown in Table F-15.
2. Compute the Wilcoxon statistic W
1 1 2 2
n
i=l W = Ei - -n(n + 1) = 178 - - (10) (11) = 123
3. Compute an approximate Z,,,. The expected value and standard deviation of W under the null hypothesis are given by the formulas
E(W) = - 1 (15) (10) = 75; 2
F-6 1
Overall ranks
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Sum
FEMP Background Study May 1994
Table F-15 Example Data Set Number 6
Total Uranium values used
in calculations (a) 0.05 0.05 0.05 0.05 0.05 0.05 0.275 0.275 0.476 0.5 0.7 0.735 0.756 0.851
1 1 1 1 1.1
1.2325 1.5 1.791 2 2
3.852
Type of
well Private Private Private Private Private FEMP Private Private Private FEMP FEMP Private Private Private Private Private Private FEMP FEMP FEMP Private FEMP FEMP FEMP FEMP
RZ Private
1 2 3 4 5
7 8 9
N/A
N/A N/A 12 13 14 15 16 17
N/A N/A N/A
N/A N/A N/A N/A
21
147 (a) These values are from Table F-14.
CIN/OU5R1/419195/TABF15.XLS/5-94 F 4 2 , I
000987
5644
FEMP Background Study May 1994
An approximate Z,,, for the Wilcoxon Rank-Sum test then follows as: -,v.- 1
4. Compare the approximate && to the upper 95'h percentile of the standard normal distribution = 1.645. Since the approximate Z,,, is greater than 1.645, the null hypothesis may be rejected at the 5 percent significance level, suggesting that there is statistically significant evidence that the populations have different means. The FEMP and private wells for Total Uranium in the Glacial Overburden are not from the same populations and not not be combined.
F-63
FEMP Background Study May 1994
1
x11
x 1 2
1 Xln
Kruskal-Wallis Test for Comparing Populations
2 3 ... k
X2l '31 ... 'k1
x22 '32 ... xk2
...
xhk ...
3 x3n
2 X2"
The Kruskal-Wallis test (Gilbert 1987) for comparing populations does not require that data sets be drawn from underlying distributions that are normal or even symmetric, but the K distributions are assumed to be identical in shape. The null hypothesis is
Ho: The populations from which the data sets have been drawn have the same means.
The alternative hypothesis is
HA: At least one population has a mean larger or smaller than at least one other population.
The data can be illustrated as follows:
The total number of data points is m = n1 + n2 + ... + n,; the q need not be equal.
Theses steps must be followed:
1. Rank the m data points from smallest to largest in which the smallest value has rank 1 and the largest value has rank m. If data points are equal (Yies") assign the midrank (e.g., data points 10 and 11 are the same; therefore, 10.5 is used for the rank of both points). If less-than-minimum detectable values occur, treat them as tied values that are less than the smallest detected value that is greater than the minimum detection limit. Suppose the m ranked values are as follows:
F-64 OQ0989
6644
0.05 0.05 0.05 0.08 0.09 0.12 0.12 0.20 0.21 0.21 0.21
Data Value I E? 1 2 3 4 5 6 7 8 9 10 11
Assigned Rank
2 2 2 4 5 6.5 6.5 8 10 10 10
FEMP Background Study May 1994
2. Compute Rj the sum of the ranks for each data set.
3. If there are no tied or less-than-minimum detectable values, compute.
where:
m Rj 9 k
= total number of data values over all data sets = sum of ranks of the jth data set = number of values in the jth data set = number of data sets
4. If there are ties or less-than-detectable values, compute:
F-65 000990
FEMP Background Study May 1994
where:
g ‘i
= number of groups with ties = the number of tied data in the jth group
5. Reject H, at a level and accept Ha if K, (g) is equal to or greater than xl*pl using a probability level of l-a and k-1 degress of freedom taken from Table F-16.
Example:
To illustrate the application of the Kruskal-Wallis & test consider the example data set presented in Table F-16. For these data, m = 30 + 30 + 21 = 81.
1. Rank all 81 observations (Table F-17).
2. The sum of the ranks for each group are as follows (Table F-18).
R, = 1454 R, = 1209.5 R, = 657.5
3. Compute:
(1454)* (1209.5)* + (657.5), ] ] - (82) l2 [ 30 21 +
81 (82) 30
= 0.001807 (70,471 + 48,763 + 20,586) - 246
= 252.61 - 246
= 6.61
5644 .: FEMP Background Study
May 1994
value 0.990 0.995 6.63 7.88 9.21 10.60 11.34 12.84 13.28 14.86 15.09 16.75 16.81 18.55 18.48 20.28 20.09 21.95 21.67 23.59 23.21 25.19
26.76 28.30 29.82 31.32 32.80 34.27 35.72 37.16 38.58 40.00
41.40 42.80 44.18
~ 48.29 49.65
1 50.99 52.34 53.67
Degrees of
heedom V 1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30
40 50 60 70 80 90 100 X
,
31.26 32.91 34.53 36.12 37.70 39.25 40.79 42.31 43.82 45.31
46.80 48.27 49.73
"5:: 54.05 55.48 56.89 58.30 59.70
Table F-16 Quantiles of the Chi-square Distribution with v Degrees o Freedom
- 0.006 0.00 0.01 0.07 0.21 0.41 0.68 0.99 1.34 1.73 2.16
2.60 3.07 3.57 4.07 4.60 5.14 5.70 6.26 6.84 7.43
8.03 8.64 9.26 9.89 10.52 11.16 11.81 12.46 13.12 13.79
20.71 27.99 35.53
51.17 59.20 67.33
-
43.28
-2.58 -
- 0.010 0.00 0.02 0.1 1 0.30 0.55 0.87 1.24 1.65 2.09 2.56
3.05 3.57 4.1 1 4.66 5.23 5.81 6.41 7.01 7.63 8.26
8.90 9.54 10.20 10.86 11.52 12.20 12.88 13.56 14.26 14.95
22.16 29.71 37.48 45.44 53.54 61.75 70.06
-
-2.33 -
Prob 0.025 0.00 0.05 0.22 0.48 0.83 1.24 1.69 2.1 8 2.70 3.25
3.82 4.40 5.01 5.63 6.26 6.91 7.56 8.23 8.91 9.59
10.28 10.98 11.69 12.40 13.12 13.84 14.57 15.31 16.05 16.79
24.43 32.36 40.48 48.76 57.15 65.65 74.22 -1.96
- -
-
3il.W a 0.05 0.00 0.10 0.35 0.71 1.15 1.64 2.17 2.73 3.33 3.94
4.57 5.23 5.89 6.57 7.26 7.96 8.67 9.39 10.12 10.85
1 1.59 12.34 13.09 13.85 14.61 15.38 16.15 16.93 17.71 18.49
26.51 34.76 43.19 51.74 60.39 69.13 77.93 -1.65
-
-
obtain 0.100 0.02 0.21 0.58 1.06 1.61 2.20 2.83 3.49 4.17 4.87
5.58 6.30 7.04 7.79 8.55 9.31 10.09 10.86 11.65 12.44
13.24 14.04 14.85 15.66 16.47 17.29 18.11 18.94 19.77 20.60
29.05 37.69 46.46 55.33 64.28 73.29 82.36
-
-1.28 -
g a va 0.250 0.10 0.58 1.21 1.92 2.67 3.45 4.25 5.07 5.90 6.74
7.58 8.44 9.30 10.17 11.04 11.91 12.79 13.68 14.56 15.45
16.34 17.24 18.14 19.04 19.94 20.84 21.75 22.66 23.57 24.48
33.66 42.94 52.29 61.70 71.14 80.62 90.13 -0.67
-
-
le of c 0.500 0.45 1.39 2.37 3.36 4.35 5.35 6.35 7.34 8.34 9.34
10.34 11.34 12.34 13.34 14.34 15.34 16.34 17.34 18.34 19.34
20.34 21.34 22.34 23.34 24.34 25.34 26.34 27.34 28.34 29.34
49.33 59.33 69.33 79.33 89.33 99.33
0
-
39.34
-
-squre 0.750 1.32 2.77 4.1 1 5.39 6.63 7.84 9.04 10.22 1 1.39 12.55
13.70 14.85 15.98 17.12 18.25 19.37 20.49 21.60 22.72 23.83
24.93 26.04 27.14 28.24 29.34 30.43 31.53 32.62 33.71 34.80
45.62 56.33 66.98 77.58 88.13 98.65 109.14 0.674
-
-
small 0.900 2.71 4.61 6.25 7.78 9.24 10.64 12.02 13.36 14.68 15.99
17.28 18.55 19.81 21.06 22.31 23.54 24.77 25.99 27.20 28.41
29.62 30.81 32.01 33.20 34.38 35.56 36.74 37.92 39.09 40.26
51.81 63.17 74.40 85.53 96.58 107.57 I 18.5C 1.282
- -
-
'than 0.950 3.84 5.99 7.81 9.49 1 1.07 12.59 14.07 15.51 16.92 18.31
19.68 21.03 22.36 23.68 25.00 26.30 27.59 28.87 30.14 31.41
32.67 33.92 35.17 36.42 37.65 38.89 40.1 1 41.34 42.56 43.77
55.76 67.50 79.08 90.53 101.s 113.15 124.34 1.645
- -
-
te tab1 0.975 5.02 7.38 9.35 11.14 12.83 14.45 16.01 17.53 19.02 20.48
21.92 23.34 24.74 26.12 27.49 28.85 30.19 31.53 32.85 34.1 7
35.48 36.78 38.08 39.36 40.65 41.92 43.19 44.46 45.72 46.98
59.34 71.42 83.30 95.02 106.62 118.14 129.56 1.96
-
-
24.73 26.22 27.69 29.14 30.58 32.00 33.41 34.81 36.19 37.57
38.93 40.29 41.64 42.98 44.31 45.64 46.96 48.28 49.59 50.89 63.69 76.15 88.38 100.4 1 12.31 124.12 135.81 2.326 -
- 0.W 10.83 13.82 16.27 18.47 20.51 22.a 24.32 26.12 27.88 29.59
-
2.576
73.40
99.61 ~ 112.3: 124.W 137.2'
~ 149.4! 3.090
186.66
- For degrees of freedom (v) > 100, chi-squared=v[l-2/9v+X *(2/9v)*0.5]^3 or chi-~quared=0.5*[X+(2v-l)~0.5]~2
Source: Table A-19, Gilbert 1987.
if less accuracy is needed, where X is given in the last row of the table.
a ;., 000992
F-67
FEW Background Study May 1994
Table F-17. Ranking of Example Data Set Number 7 (Concentrations in pg/L)
Rank Group ChemicalA Rank Group ChemicalA
!I 1 2 3 4 5
6 7 8 9 10 11 12 l3 14 15 16 175 175) 19 20 21 22 23 24 25 265
28 29 30 31 32 33 34 35 36 37 38 39 405
42
45)
263
405)
1 2 3 1 2 2 3 3 3 2 1 3 3 3 2 2 2 2 1 2 3 3 1 2 2 2 3 3 1 1 1 1 3 3 3 3 1 2 3 2 2 1
189 251 288 301 301 303 304 338 345 348 351 352 356 358 360 364 366 366 370 371 372 378 386 387 405 406 406 420 422 437 451 456 459 464 467 472 481 482 483 486 486 488
43 445
46 47 48 49 50 51 52 53 54 555
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81
445)
555)
1 1 2 1 1 3 2 2 1 2 1 1 2 2 2 2 3 2 1 3 1 1 2 2 1 3 1 2 2 1 1 1 1 2 2 1 1 3 1
521 534 534 535 543 560 561 566 581 583 615 619 621 621 654 667 681 745 747 759 766 785 812 829 840 931 941 1020 1040 1050 1070 1090 1150 uoo 1410 1460 1500 1750 4850
Group 1 = Dry Fork Group 2 = Ross Group3 = Shandon
) = ties \
FEM
P Ba
ckgr
ound
Study
May
1994
Tab
le F
-18.
Ran
king
of E
xam
ple
Dat
a Se
t Num
ber
7 by G
roup
(C
once
ntra
tions
in p
g/L
)
Dry
For
k R
oss
Shan
don
1 2
3 4
5
45
7
11
6 8
19
10
9 23
15
12
29
16
13
30
17.5
14
31
17
.5
21
32
20
22
37
24
265
42
25
28
43
265
33
445
38
34
46
405
35
47
405
36
51
445
39
53
49
48
54
50
59
61
52
62
63
555
68
64
555
80
67
57
69
58
72
60
73
65
74
66
75
70
78
71
79
76
81
n SUm
1454
12
095
6575
cEN
/OU
SRI/
wp/
4 I9
1%/A
F'PE
ND
IXF/
SW
-/
.
/
F-69
O
Q0
994
EM
F' Ba
ckgr
ound
Stu
dy
May
199
4
4.
Bec
ause
ther
e ar
e tie
s, ca
lcul
ate
a m
odifi
ed c
:
--
I
1-
1
E 'j (
f -
1)
M
(m2
- 1)
j=
1
whe
re: g tl t2 t3 t4 tS t6
= 6
= n
umbe
r of
grou
ps w
ith t
ies
= 2
= n
umbe
r of
data
poi
nts
in th
e fir
st tl
zd g
roup
(R
ank -3
) =
2 =
num
ber o
f da
ta p
oint
s in
the
seco
nd ti
ed g
roup
(R
ank
17.5
) =
2 =
num
ber o
f da
ta p
oint
s in
the
third
tie
d gr
oup
(Ran
k 26
.5)
= 2
= n
umbe
r of
data
poi
nts in
the
four
th ti
ed g
roup
(R
ank
46.5
) =
2 =
num
ber
of d
ata
poin
ts in
the
fifth
tied
gro
up (R
ank
44.5
) =
2 =
num
ber
of d
ata
poin
ts in
the
sixth
tied
gro
up (R
ank
55.5
) ,
6.61
1-
1
[ 2(3
) +
2(3
) +
2(
3) +
2(
3) +
2(
3) +
2
~1
81
[(8
1)2 -
1)
- 6.
61
- 0.
9999
3
= 6.
61
5.
Bec
ause
=
6.6
1 is
grea
ter t
han
= 5
.99,
the
null
hypo
thes
is H
,, is
reje
cted
at
the a
= 0
.05 le
vel.
The
con
clus
ion
is th
at a
t lea
st o
ne p
opul
atio
n be
ing
com
pare
d ha
s a
mea
n di
ffer
ent f
rom
the
othe
r pop
ulat
ions
.
F-70