fraud detection system

Type I error rates and test power for some variance components estimation methods: one-way random effect model

Mehmet Mendeş

Biometry and Genetics Department, Animal Science Department, Faculty of Agriculture, Çanakkale Onsekiz Mart University, Çanakkale, Turkey

AbstractThis study was conducted to compare Type I error and test power of ANOVA, REML and ML methods by Monte Carlo simulation technique under different experimental conditions. Simulation results indicated that the variance ratios, sample size and number of groups were important factors in determining appropriate methods which were used to estimate variance components. The ML method was found slightly superior when compared to ANOVA and REML methods. On the other hand, ANOVA and REML methods generated similar results in general. As a results, regardless of distribution shapes and number of groups and if n<15; ML, REML methods might be preferred to the ANOVA. However, when either number of groups or sample size was increased (n≥15) ANOVA method may also be used along with ML and REML.

Keywords: variance components, ANOVA, REML, ML, type I error rate, test power

IntroductionEstimation of the amount of variation between treatments in total variation is an important issue for animal breeding as well as other fields of science (Falconer 1989). Variance components estimation methods such as Analysis of Variance (ANOVA), Maximum Likelihood (ML), Restricted Maximum Likelihood (REML), Bayesian, Henderson I, II, III are the ways to evaluate the amount of variation in response variable which is related to one or more random-effects factors (Saahai & Ojeda 2004, 2005). ANOVA method has been exploited by many years in estimating variance components (Searle et al. 1992, Yolcu et al. 2004). However, even if the assumptions of ANOVA (normality and homogeneity of variance) are satisfied, it tends to predict negative variance component when mean square treatment is smaller than mean square error (MSA<MSE) (Verdooren 1982, Smith & Sawage 1992, Kelly & Mathew 1993, 1994, Rao 2001, Searle et al. 2006). Searle (1971) reported that when MSA<MSE, there is no mechanism used in ANOVA that prevents a negative estimate. It is possible, however, to avoid getting negative estimates by using some other methods such as ML, REML, and Bayesian method instead of ANOVA. Searle et al. (2006) reported that there is nothing in the ANOVA method that will prevent a negative estimate. Another alternative would be to use a method of estimation that explicitly excludes the possibility of negative estimates such as ML, REML and Bayesian method.

Archiv Tierzucht 55 (2012) 5, 506-518, ISSN 0003-9438© Leibniz Institute for Farm Animal Biology, Dummerstorf, Germany

Arch Tierz 55 (2012) 5, 506-518 507

There are many studies about comparing variance component methods with respect to their performance on both a real data set and an artificial data set generated by simulation techniques (Swallow & Monaham 1984, Rao & Kleffe 1988, Westfall 1987, Chaloner 1987, Khattree & Gill 1988, Smith & Sawage 1992, Westfall & Bremer 1994, Kelly & Mathew 1994, Belzile & Angers 1995, Rao 2001, Shugart et al. 2002). However, these methods were compared to each other only based on one data set. It does not provide efficient comparison of the performances of these methods based on one data set. Comparison of the performance of these methods with respect to Type I error rate and test power under different experimental conditions made by Monte Carlo simulation studies, might provide more detailed information. The main purpose of this study was to compare ANOVA, ML and REML methods with respect to Type I error rates and test power under different experimental conditions using Monte Carlo simulation studies.

Material and methodsMonte Carlo simulation techniques were used to compare ANOVA, ML and REML methods with respect to Type I error rates and test power under different experimental conditions. Type I error rates and test power of these methods were assessed under different situations. The following variables were manipulated in this simulation study:

a) type of populationb) number of groupsc) sample sizes d) variance homogeneity and heterogeneitye) effect sizes.

For k=3 and k=5, random numbers from the N (0,1), χ2 (3), t (10) and β (5,2) distributions were generated by a Microsoft Fortran Developer Studio (Microsoft Corporation, Redmond, WA, USA) using generators from IMSL library (functions RNNOA, RNSTT, RNCHI and RNBET) (Anonymous 1994). The variance ratios were chosen as 1:1:1, 1:1:4, 1:1:9 and 1:1:25 for k=3 and 1:1:1:1:1, 1:1:1:1:4, 1:1:1:1:9 and 1:1:1:1:25 for k=5. In order to form heterogeneity among the population variances, random numbers in the samples were multiplied by specific constant numbers (σ=1, 2, 3, 5). The populations were standardised because they have different means and variances. Shape of distributions was not changed while the means were changed to 0 and the standard deviations were changed to 1. For k=3 and k=5, 30 000 data sets were generated from mentioned populations employing equal sample sizes of 5, 10, 15 and 20.

For each experimental condition, the ANOVA, ML and REML methods were performed to estimate variance components (σ2

e and σ2α), respectively. Then, the numbers of predicted σ2

α

s whose values fall outside of the confidence interval was calculated as follows and counted afterwards. And then, dividing the value of σ2

α whose values fall outside of the confidence interval by the total number of 30 000 trials, type I error rates were found. Since there is a solution only for σ2

α>0 in terms of ML and REML methods, the negative predicted values of σ2α

s were assumed to be zero (Sahai & Ojeda 2005):

P MSE ( MSA −F(k−1, k(n−1); 1−α2 ) )< σ2

α< MSE ( MSA −F(k−1, k(n−1); α2 ) ) = 1 − α (1)

nF(k−1, ∞; 1−α2 ) MSE nF(k−1, ∞;

α2 ) MSE

Mendeş: Type I error rates and test power for some variance components estimation methods508

In order to compare the methods with regard to test power, firstly, specific constant numbers with standard deviation form (effect sizes of 0.50, 0.75 and 1.25) were added to the random numbers of the last population. Thus, the predicted values of σ2

α s were provided to fall outside of the confidence interval in question. Then, the number of σ2

α s falling outside of the intervals which were originally falling outside was determined. Finally, having divided that number by 30 000 which is the total number of trial, the power of the test was obtained.

Statistical methods

If there are k treatment groups and each group has the same number of observations (balanced design), the general form of the random-effects one-way model can be written as below:

yij = μ + αi + eij , i = 1, 2, …, k and j = 1, 2, …, n (2)

where yij is the observed value of experimental unit j in group i, μ is the general population mean, αi is the effect of group i, eij is the random error terms with (0, σ2

e). The eij’s and αi’s are mutually exclusive and αi ~ N(0, σ2

α), eij ~ N (0, σ2e) and Cov (αi , eij)=0

(Rasch et al. 1999, Kaps & Lamberson 2004, Rasch & Masata 2006).In random-effects model, the expected value and variance of yij is μ and σ2

α+σ2e), respectively.

Since αi and eij are mutually exclusive, therefore the distribution of yij is yij ≈ N (μ, σ2α+σ2

e)The two components σ2

α and σ2e of yij are called variance components. Different methods

such as ANOVA, ML, REML, Bayesian, Henderson I, II, III, and MINQUE etc. have been developed in order to estimate the components σ2

α and σ2e. ANOVA, ML and REML methods were taken

into consideration in this study.

ANOVA method

ANOVA method for estimating σ2α and σ2

e includes equating mean square error (MSE) and mean square treatment (MSA) to their expected values and then solving the resulting equating for σ2

α and σ2e. So, the estimators obtained are as follows:

E (MSE) = σ2e and E (MSA) = σ2

e + nσ2α then (3)

^σ2e = MSE and ^σ2

a = (MSA − MSE)

n(Thompson 1962, Kelly & Mathew 1993, Rao 2001, Sahai & Ojeda 2004, 2005, Searle et al. 2006).

Under normality assumption, the probability of negative estimate for ^σ2α can be computed

for various numbers of treatments (k) and sample sizes (n) as:

σ2e (4)

σ2e + nσ2

α

Maximum likelihood method (ML)

In order to avoid negative estimates, instead of ANOVA, one of the alternative methods which is used to estimate variance components, is maximum likelihood (ML). ML estimators of σ2

e and σ2α are:

P ( ^σ2α

< 0) = P (MSA < MSE) = P (Fk-1, N-k < )

Arch Tierz 55 (2012) 5, 506-518 509

where k is the number of group, n is the number of observation, SST is the total sum of squares, MSE is the mean square error and MSA is the mean square treatment. Since ML estimators must satisfy σ2

e and σ2α ≥0, If MSA<MSE then ^σ2

α = 0 and ^σ2e = SST (Hartley & Rao 1967,

Harville 1977, Kaps & Lamberson 2004).

REML method

In order to avoid negative estimates another alternative method is REML. Under the normality assumption, for balanced one-way random model the REML estimators of σ2

e and σ2α are:

^σ2e = SSE = MSE and ^σ2

α = (MSA−MSE) respectively. (6)

These estimators are the REML estimates provided being nonnegative. Note that REML estimators are the same as the ANOVAs when studied with balanced design.

Likewise ML estimators, REML estimators must also satisfy σ2e and σ2

α ≥0, If MSA<MSE then ^σ2

α = 0 and σ2e = STT (Patterson & Thompson 1974).

ResultsType I error rates: three groups

Empirical Type I error results of 30.000 simulation runs are given in Table 1-4. When k=3, the distribution was normal (0, 1) and variances were equal (1:1:1), REML and ML methods had similar and acceptable Type I error rates (around 5.0 %). Type I error rates for ANOVA method were between 2.29-4.62 % under the same conditions. The ANOVA appeared to be the most affected method by the sample size. Under small deviations from homogeneity of variances (1:1:4) the most reliable results were obtained from the ANOVA method. Increasing the degrees of the heterogeneity of the variances (1:1:9 and 1:1:25) caused the Type-I error rates in all methods that biased more than 5.0 %.

Table 1Empirical type I error rates (%) when distributions were N (0,1)

Sample size

Varianceratiotests

1:1:1(1:1:1:1:1)

1:1:41:1:1:1:4

1:1:91:1:1:1:9

1:1:251:1:1:1:25

k=3 k=5 k=3 k=5 k=3 k=5 k=3 k=5

5

10

15

20

ANOVAMLREMLANOVAMLREMLANOVAMLREMLANOVAMLREML

2.295.334.423.635.644.794.105.674.954.625.825.24

4.755.194.984.905.305.024.825.184.925.095.345.14

3.457.055.775.047.116.355.467.316.425.516.866.24

7.287.827.577.648.097.807.688.057.817.507.867.57

5.219.618.166.769.268.376.959.128.137.338.818.11

11.8112.4812.2311.8912.3912.0911.9312.5012.1111.9512.4312.11

7.0212.6611.138.5911.6710.709.14

11.5710.619.55

11.5310.76

20.3921.4021.1120.1420.8520.5119.6720.3419.9219.7920.4220.00

kn

nN−k

kn−1

^σ2e = SST = MSE and ^σ2

α = (1− 1k (MSA−MSE) respectively. (5)

N−k n


When distributions were not normal but the variances were homogeneous, the Type I error rates for all methods were generally similar to the Type-I error rates realised under normal distribution. It may be suggested that as long as the variances are homogeneous the distribution shape does not affect Type-I error rates of all methods. However it is not possible to reach the same conclusions when the variances are not homogeneous. The Type I error rates for all methods were affected negatively when the distributions were not normal and variances were not homogeneous. Moreover, this negative effect was pronounced more when samples were taken from χ2 (3)-distribution.

If we make a general evaluation for k=3, when the variances are equal, regardless of the shape of the distribution, as the number of observed values in groups increases, the Type-I error rates regarding all methods, have a tendency to tend to gather around 5.0 %.

Five groups

Type-I error rates concerning all methods were displayed to be affected from an increase the number of groups to be compared from 3 to 5. This effect appeared slightly positive when the variances were homogeneous. Because under this experimental conditions, Type-I error rates with regard to all methods were gathered generally around 5.0 %. In case of the variances becoming heterogeneous, however, the Type I error rates regarding all methods, have a tendency to deviate from 5.0 %. Especially, when the variance ratios were 1:1:25, the Type I error rates have raised even over 20.0 %.

Under the same conditions, when the distributions were t (10) and β (5,2), the Type I error rates for all methods were generally around 5.0 % under homogeneity of variances. When the distributions were χ2 (3), it has been observed that Type I error rates were generally a bit less than 5.0 %. However, in parallel to the increase in sample size it shows a trend approaching to 5.0 %.

When the variances were heterogeneous, all methods tended to deviate from nominal alpha level (5.0 %). These deviations appeared to be more obvious especially in the skew distributions (χ2 and β). Therefore, all methods were adversely affected when both homogeneity of variance and normality assumptions were not satisfied. At the same time, increasing the number of groups to be compared, had seriously affected the type I error rates especially when the variances were heterogeneous. This effect was observed to be more obvious when the variance ratios were 1:1:25.

Test power

Empirical test power results of 30 000 simulation runs are given in Table 5-8. When samples were taken from normally distributed populations (Table 5), the ML method was slightly more powerful when compared to ANOVA and REML regardless of the number of groups, sample sizes, mean differences and variance ratios in general. When variances were homogeneous (1:1:1), the test power values for ANOVA, REML and ML methods were estimated between 5.94-96.69 %, 6.84-96.69 % and 7.63-97.05 % for k=3. Under these conditions, the test power estimates for ANOVA and REML methods are almost the same. All methods were affected from heterogeneity of the variances negatively and this effect was more obvious especially when variance ratios were 1:1:9 and 1:1:25. When the variances were heterogeneous, increasing the number of groups from 3 to 5 has increased test power estimates of all methods regardless of distribution shapes (especially when distributions were Normal, t- and Beta) and sample sizes (especially when n≥15).

Arch Tierz 55 (2012) 5, 506-518 511

Table 2Empirical type I error rates (%) when distributions were β (5,2)

Sample size

Varianceratiotests

1:1:1(1:1:1:1:1)

1:1:41:1:1:1:4

1:1:91:1:1:1:9

1:1:251:1:1:1:25

k=3 k=5 k=3 k=5 k=3 k=5 k=3 k=5

5

10

15

20


2.374.934543.705.404.854.015.484.814.315.554.95

4.544.924.774.815.174.914.915.265.015.025.295.08

4.046.996.515.197.056.395.517.116.445.757.036.35

7.307.787.567.918.328.077.537.957.657.547.887.62

6.099.759.137.149.488.687.299.218.477.388.768.16

12.8013.5713.2812.2012.7912.4812.3012.7912.5211.9712.4012.08

8.4913.5612.929.46

12.4711.819.93

12.3311.4910.1711.9911.29

22.0223.0722.8020.8721.5821.2120.2220.8520.5020.0420.5520.26

Table 3Empirical type I error rates (%) when distributions were t (10)

Sample size

Varianceratiotests

1:1:1(1:1:1:1:1)

1:1:41:1:1:1:4

1:1:91:1:1:1:9

1:1:251:1:1:1:25

k=3 k=5 k=3 k=5 k=3 k=5 k=3 k=5

5

10

15

20


2.094.594.093.545.354.744.185.695.054.325.324.93

4.655.034.874.404.704.494.745.074.874.685.024.74

3.286.035.544.766.545.805.296.966.175.546.726.15

6.997.517.296.927.367.087.357.737.457.357.717.42

4.528.097.385.958.087.436.608.557.797.278.708.01

11.1011.8211.5611.4211.9611.6411.5611.9711.6711.6512.1111.74

6.42 11.17 10.36 8.43 11.67 10.81 8.7311.3110.48 8.7310.60 9.85

19.3420.2819.9719.4220.1119.7719.7120.3620.0019.5020.0419.73

Table 4Empirical type I error rates (%) when distributions were χ2 (3)

Sample size

Varianceratiotests

1:1:1(1:1:1:1:1)

1:1:41:1:1:1:4

1:1:91:1:1:1:9

1:1:251:1:1:1:25

k=3 k=5 k=3 k=5 k=3 k=5 k=3 k=5

5

10

15

20


1.833.923.523.214.884.263.745.104.554.325.354.86

3.944.274.124.234.544.364.444.784.524.544.854.61

4.056.976.365.597.506.835.557.066.326.177.376.84

6.827.357.107.387.817.507.788.177.877.798.167.89

7.2210.499.718.29

10.599.808.48

10.449.658.379.829.25

12.9413.6513.3112.5813.1012.8112.5913.1412.7912.5513.0712.69

10.4215.0214.2711.3614.4013.5810.9213.3612.6011.1513.0012.32

21.8222.6822.4322.3323.1522.7520.9121.5321.2120.7821.3821.00


Table 5Empirical test powers (%) when distributions were N (0,1)

n=5:5:5 / 5:5:5:5:5

Variance ratioTests Effect size 1:1:1

(1:1:1:1:1)1:1:4

1:1:1:1:41:1:9

1:1:1:1:91:1:25

1:1:1:1:25k=3 k=5 k=3 k=5 k=3 k=5 k=3 k=5

ANOVA

ML

REML

ANOVA

ML

REML

ANOVA

ML

REML

ANOVA

ML

REML

0.500.751.250.500.751.250.500.751.25

5.9410.0428.347.63

11.7230.876.84

10.5828.43

6.379.97

27.067.16

11.0229.186.55

10.0427.08

6.558.66

16.738.42

10.6618.887.789.7917.39

8.9711.7820.519.8212.7421.839.20

12.0120.61

7.609.17

13.709.96

11.5315.999.34

10.8114.93

13.0614.6119.8514.0115.6521.1013.4414.9820.20

9.2610.3612.0712.5613.3415.1711.8912.6314.32

20.7920.8722.8022.0522.2224.1721.4221.5823.48

n=10:10:10 / 10:10:10:10:100.500.751.250.500.751.250.500.751.25

11.6124.1867.4013.0126.2769.6411.9124.2967.41

10.5722.1966.8211.4723.6668.5210.6422.2066.82

9.7015.3635.4011.0016.8437.2110.2915.7935.55

11.5718.2940.2312.3519.2341.6911.7018.3740.25

9.7412.7722.1511.3214.1323.8110.6714.0022.62

11.5017.9629.6015.3318.8130.6914.7318.1429.67

10.3111.5415.3512.0913.4217.1811.6012.7216.43

21.2122.0325.6622.0322.9126.6221.5822.4025.97

n=15:15:15 / 15:15:15:15:150.500.751.250.500.751.250.500.751.25

17.7640.3788.6919.2742.5289.7417.9240.3988.69

15.6937.6488.8816.8039.3589.6615.7037.6588.88

12.2421.6551.6113.5423.2253.3912.6421.8951.65

14.7625.3657.6415.5226.4558.8214.8725.4157.64

10.8915.8331.6912.1017.0433.4011.4616.1931.89

15.9821.5639.3516.7022.3440.5516.1221.6739.41

11.0412.7518.7012.5714.2720.1712.0413.6419.34

21.3322.7128.9922.0923.4629.8421.6622.9229.17

n=20:20:20 / 20:20:20:20:200.500.751.250.500.751.250.500.751.25

23.7554.7296.6925.3356.6797.0523.8354.7496.69

21.1352.3997.0522.5554.1597.3321.1552.3997.05

14.7128.1566.1415.9629.6967.6414.9328.2866.14

17.7632.4871.2018.5133.5472.2017.8032.5071.19

12.3219.0940.6813.3820.4842.2512.7119.4740.79

17.6024.8148.4518.2225.6449.5017.6824.9248.47

11.6813.7422.3312.8515.0523.5812.3114.3622.74

21.2623.8832.9721.9124.6033.8721.5124.0733.09

The test power estimates for ANOVA and REML were almost the same in general regardless of experimental conditions. Deviations from homogeneity of variances had a significant effect on the test power estimates. When the deviations from homogeneity of variances were obviously large (1:1:25), the maximum test power estimates for ANOVA, ML, and REML were observed as 22.33 %, 23.58 % and 22.74 % for k=3 and 32.97 %, 33.87 % and 33.09 % for k=5, respectively. Under the homogeneity of variances, when n≥15 and Δ=1.25, each method

Arch Tierz 55 (2012) 5, 506-518 513

has reached the power level of 80.0 % which was accepted as an adequate test power value regardless of distributions. However, none of the methods could reach 80 % test power values even if n=20 and Δ=1.25 under heterogeneity of variances. For example, when distributions were normal and variance ratios were 1:1:4, n=20 and Δ=1.25; the estimated test power values for ANOVA, ML and REML were between 14.71-66.14 %, 15.96-67.64 % and 14.93-66.14 % for k=3 and 17.76-71.20 %, 18.51-72.20 % and 17.80-71.19 % for k=5 respectively.

Table 6Empirical test powers (%) when distributions were t (10)

n=5:5:5 / 5:5:5:5:5


(1:1:1:1:1)1:1:4

1:1:1:1:41:1:9

1:1:1:1:91:1:25

1:1:1:1:25k=3 k=5 k=3 k=5 k=3 k=5 k=3 k=5

ANOVA

ML

REML

ANOVA

ML

REML

ANOVA

ML

REML

ANOVA

ML

REML

0.500.751.250.500.751.250.500.751.25

0.500.751.250.500.751.250.500.751.25

5.768.49

21.866.479.45

23.735.938.61

21.88

5.867.91

14.687.669.87

16.837.089.07

15.47

8.3610.7518.379.18

11.7619.668.62

10.9818.52

7.178.3611.989.42

10.7314.298.82

10.0013.33

12.1413.8917.7813.0714.9918.8912.5114.3018.03

9.009.6212.1112.1012.6814.3211.4412.0113.34

18.9619.6621.3520.1720.9422.5919.6120.3121.97

n=10:10:10 / 10:10:10:10:100.500.751.250.500.751.250.500.751.25

9.8620.0456.6511.2621.8258.9710.2620.2256.66

9.1117.8855.529.87

19.2357.379.16

17.9155.51

8.7313.4729.8910.0315.0031.739.28

13.9630.13

10.8215.7734.5911.5716.7035.9410.9515.8634.63

9.0411.3220.1610.5612.7621.779.99

12.0420.61

13.7316.7926.5514.5117.6027.6113.9816.9526.70

10.1211.3914.1511.9213.1416.1711.3912.5615.32

20.3721.5524.6721.3422.4225.5720.7421.8924.96

n=15:15:15 / 15:15:15:15:150.500.751.250.500.751.250.500.751.25

14.3032.8979.6215.6334.9081.1414.4932.9279.92

12.3329.6279.1713.3131.2480.4512.3529.6379.16

11.0518.5644.2112.2119.9446.0411.4718.8244.28

13.2421.5449.2013.8922.5450.4713.3221.5849.20

9.6914.2927.6010.8015.4529.0910.2214.7827.82

15.1419.5934.6115.8020.3835.5615.3019.7234.68

10.6611.8117.1211.9513.3018.6211.4812.6417.81

20.6022.0127.4321.2922.7628.3020.8322.2227.62

n=20:20:20 / 20:20:20:20:200.500.751.250.500.751.250.500.751.25

19.3644.6191.3720.9246.5792.0919.4444.6491.37

17.0841.8691.9218.2143.5692.5017.0941.8791.93

12.8023.9056.2513.9325.3657.9613.0924.0756.28

15.8327.4361.5216.5228.4662.6615.8927.4661.53

11.9116.8834.3512.9218.1135.7312.3317.2234.48

16.1922.8142.0316.8623.5943.0616.3122.9242.07

11.3313.2519.8512.4714.5121.3111.9813.8720.24

20.9423.0930.4021.6223.8031.2721.1223.3130.59


When the variance ratios increased to 1:1:9 and 1:1:25, the test power estimates for ML method were slightly higher than those of the ANOVA and REML. Under these conditions, the maximum test power values were estimated as 42.25 % and 23.58 % for k=3, and 49.50 % and 33.87 % for k=5, respectively. The similar results were obtained for other distributions (χ2, t and β). ML method was also slightly powerful when compared to ANOVA and REML when the samples were taken from the χ2, t and β distributions.

Table 7Empirical test powers (%) when distributions were β (5,2)

n=5:5:5 / 5:5:5:5:5


(1:1:1:1:1)1:1:4

1:1:1:1:41:1:9

1:1:1:1:91:1:25

1:1:1:1:25k=3 k=5 k=3 k=5 k=3 k=5 k=3 k=5

ANOVA

ML

REML

ANOVA

ML

REML

ANOVA

ML

REML

ANOVA

ML

REML

0.500.751.250.500.751.250.500.751.25

5.839.94

27.407.05

11.0328.676.67

10.4227.47

6.319.68

26.526.95

10.6828.646.379.77

26.53

5.536.87

12.677.038.22

13.856.767.90

13.25

7.929.36

16.908.59

10.2418.178.109.5717.00

7.067.409.628.959.10

11.028.708.82

10.67

11.4412.1215.0112.4213.1216.0411.8512.5115.24

9.349.389.55

12.2512.2011.8812.0011.9411.55

20.2419.7819.3421.4620.9720.5520.9420.4719.93

n=10:10:10 / 10:10:10:10:100.500.751.250.500.751.250.500.751.25

11.3524.3467.6112.4725.7769.2611.6924.4667.61

10.1921.7466.8311.0423.2368.6410.2221.7566.83

7.8912.5632.408.89

13.6833.718.41

12.9132.46

10.4315.4037.8211.2016.3739.2710.5615.4837.83

8.119.6418.129.28

10.9219.458.86

10.3518.47

12.3914.8725.2513.1215.6426.2012.5715.0125.32

9.249.9812.1010.8811.6013.5410.4811.1813.03

19.4519.7121.8420.3720.6022.7219.8420.0522.10

n=15:15:15 / 15:15:15:15:150.500.751.250.500.751.250.500.751.25

17.0639.3889.0318.3741.0589.9417.2239.4089.03

15.4237.1889.4216.4738.9890.2315.4537.1989.42

10.3019.1251.2711.3620.2752.8710.7219.2451.29

12.8222.6556.9913.5523.7458.4712.8722.6956.99

9.1812.8628.1210.3014.0029.419.80

13.2828.29

13.7517.9835.7014.4018.7536.9113.8618.0735.74

10.1410.7415.1711.3511.9416.4511.0011.4515.73

19.5920.5125.0020.2721.1725.7419.8620.7525.16

n=20:20:20 / 20:20:20:20:200.500.751.250.500.751.250.500.751.25

23.6654.5596.7725.0956.4397.1023.7254.5696.77

20.6652.2497.3221.9653.9297.6120.6752.2497.32

12.8425.8166.8813.8527.1668.3413.0525.9266.89

15.7630.3872.1216.4931.4373.1715.8130.3972.13

10.3515.8938.0411.3716.9739.4610.7716.1838.12

15.4621.9246.3816.1022.6747.4915.5721.9746.41

10.3412.0818.3811.4513.1719.7011.0212.6818.79

19.8621.8928.5220.5222.5829.3520.0622.0128.64

Arch Tierz 55 (2012) 5, 506-518 515

The test power values for ANOVA and REML were also found to be quite similar under these conditions. On the other hand, in parallel to increase in sample size the test power estimates for all methods were close to each other.

The test power estimates for all methods under β (5,2) distributions were similar to those of the estimates under normal distributions especially when the variances were homogenous (1:1:1) or the deviations from homogeneity of variances were small (1:1:4).

Table 8Empirical test powers (%) when distributions were χ2 (3)

n=5:5:5 / 5:5:5:5:5


(1:1:1:1:1)1:1:4

1:1:1:1:41:1:9

1:1:1:1:91:1:25

1:1:1:1:25k=3 k=5 k=3 k=5 k=3 k=5 k=3 k=5

ANOVA

ML

REML

ANOVA

ML

REML

ANOVA

ML

REML

ANOVA

ML

REML

0.500.751.250.500.751.250.500.751.25

6.2511.9433.947.30

12.9935.137.00

12.3534.00

5.9610.1430.826.6611.2833.086.0510.1930.83

3.875.3011.254.976.35

12.284.826.12

11.66

6.076.92

14.626.787.59

15.896.307.04

14.70

5.825.296.857.606.828.097.376.567.72

9.328.95

10.8310.159.7111.769.669.2211.02

10.089.267.91

12.5511.589.78

12.2511.319.48

18.6717.0014.7219.8818.1415.7719.2917.5615.21

n=10:10:10 / 10:10:10:10:100.500.751.250.500.751.250.500.751.25

12.4727.1969.8113.6028.6271.1812.7627.2569.81

10.7424.0168.3011.6625.5769.9110.7924.0368.30

6.3010.1432.407.19

11.2134.076.76

10.4832.49

8.1413.3537.538.78

14.2539.148.23

13.4037.53

6.637.44

14.587.788.51

15.677.378.01

14.77

9.6110.9821.4610.3511.6322.619.8011.0621.50

9.508.679.20

11.0610.1510.5910.699.78

10.05

17.6916.7617.1218.4617.5618.0618.0017.1017.39

n=15:15:15 / 15:15:15:15:150.500.751.250.500.751.250.500.751.25

18.7442.8588.4020.0644.7289.2718.8742.8788.40

15.5239.2189.1116.5440.9390.0015.5439.2189.11

8.1716.8053.079.09

18.0754.898.49

16.9553.10

10.8220.7059.0911.4621.7760.5710.8620.7359.09

7.1910.0925.618.13

11.2127.107.67

10.4625.73

11.3214.5334.0311.9115.3535.2111.4114.5934.06

9.448.8911.5810.6010.0312.8010.259.63

12.05

17.4317.0520.8118.1117.8621.6417.6617.3020.92

n=20:20:20 / 20:20:20:20:200.500.751.250.500.751.250.500.751.25

24.9956.4696.0926.5958.1296.4425.0756.4696.09

21.5453.4197.0122.8755.1197.3021.5453.4297.02

10.8024.5669.9111.6626.0271.5611.0724.6469.92

13.8028.5875.0214.5429.8476.1013.8328.6075.01

8.1012.7736.518.91

13.8638.258.47

13.0436.59

12.5518.4846.3213.1219.4747.4612.6318.5346.32

9.149.58

14.7210.1910.5515.859.82

10.0015.01

17.7918.0324.8018.4418.7525.7718.0018.1824.90


Therefore, when the variances were homogeneous or small deviations occurred from the homogeneity, having normal distributions or β(5,2) did not affect the performances of these methods.

However, the effect of the distribution shape on test power became more apparent as the variance ratios were increased. Under these experimental conditions, the ML method was found to be slightly superior when compared to the ANOVA and REML when distributions were normal (0, 1) and t (10). On the other hand, in parallel to the increase in sample sizes, the test powers get close to each other.

When the samples were taken from χ2 distributions and variances were heterogeneous, the test power estimates for all methods were smaller than those of the normal (0,1), t (10) and β(5,2). Especially the test power estimates were affected from high heterogeneity in the variances (1:1:25). For example, the test power estimates for ANOVA, ML and REML were 14.72 %, 15.85 % and 15.01 % for k=3 even if n=20 and Δ=1.25. Under the same conditions, the test power estimates for β(5,2) were 19.85 %, 21.31 %, 20.24 %, the test power estimates for t(10) 19.85 %, 21.31 %, 20.24 % and for normal (0,1) populations were 22.33 %, 23.58 % and 22.74 %, respectively. Under the same conditions when k=5, the test power estimates for these methods were 24.80 %, 25.77 % and 24.90 % for χ2(3), 28.52 %, 29.35 % and 28.64 % for B(5,2), 30.40 %, 31.27 %, 30.59 % for t(10) and 32.97 %, 33.87 % and 33.09 % for normal (0,1) populations respectively.

DiscussionThe estimation of variance components methods are commonly used in population genetics and applied in animal breeding to estimate the amount of variation between treatments in total variation and to estimate some breeding values (Rash & Masata 2006). For this purpose different methods were developed. In this study the ANOVA, ML and REML methods commonly used in practice, were compared to each other according to their performances (Type I error rate and test power) under 384 different experimental conditions. Results of this study showed that the ML method showed a slightly higher performance when compared to REML and ANOVA in general. On the other hand, ANOVA and REML estimates were found similar when variances were homogeneous. However, in parallel to the increase in the sample size (especially n≥15), both the Type-I error rates and the power values of the test of all the methods have gradually approached to each other. Searle (1971) and Searle et al. (2006) reported that the REML estimates were usually less biased when compared to ML estimates. For this reason, the REML estimates were generally preferred to ML and ANOVA in practice. Yang & Su (2005) reported that the performances of the ML, REML and ANOVA estimators became very similar when sample sizes were large enough. On the other hand, they could all perform poorly when there was an unbalanced design with large variances. Results of this study also showed that all methods were affected adversely from heterogeneity of variances. But, the existence of lower level (1:1:4 and 1:1:1:1:4) deviations from the homogeneity of variances did not affect the Type I error and test power estimates. When a general evaluation was made, it is possible to assert that regardless of the shape of the distribution, if the variances were homogeneous, ML and REML methods were superior to ANOVA. Under the same conditions, on the other hand, if the variances were not homogenous, the Type

Arch Tierz 55 (2012) 5, 506-518 517

I error rate and test power of the ANOVA, ML, and REML were at very low levels. Therefore, ANOVA, ML, and REML estimates would be biased under these conditions. Sample size and number of groups are also important factors in determining appropriate methods which were used to estimate variance components. Past studies indicated that the probability of negative variance component estimate decreases as either k or n increases (Searle et al. 2006). Regardless of distribution shapes and number of groups, if n<15 ML and REML methods might be preferred to the ANOVA. However, when either the number of groups or sample size was increased (n≥15), the ANOVA method may be used along with ML and REML. Because as the sample sizes were increased, the numbers of negative component estimates were decreased.

ReferencesAnonymous (1994) FORTRAN subroutines for mathematical applications. IMSL Math/Library. Visual Numerics,

Houston, USA

Belzile E, Angers JF (1995) Choice of noninformative priors for the variance components of an unbalanced one-way random model. Commun Stat Theory Methods 24, 1325-1341

Chaloner K (1987) A Bayesian approach to the estimation of variance components for the unbalanced one-way model, Technometrics 29, 323-337

Falconer DS (1989) Introduction to quantitative genetics. 3rd rev. ed. Wiley, New York, USA

Hartley HO, Rao JNK (1967) Maximum-likelihood estimation for the mixed analysis of variance model. Biometrica 54, 93-98

Harville DA (1977) Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems, J Am Stat. Assoc 72, 320-338

Kaps M, Lamberson WR (2004) Biostatistics for animal science. CABI Publ., Wallingford, UK

Kelly RJ, Mathew T (1993) Improved estimators of variance-components with smaller probability of negativity. J R Stat Soc Series B 4, 897-911

Kelly RJ, Mathew T (1994) Improved nonnegative estimation of variance components in some mixed models with unbalanced data. Technometrics 36, 171-181

Khattree R, Gill DS (1988) Comparison of some estimates of variance components using Pitman’s nearness criterion. ASA Proceedings of Statistical Computing Section, 133-136

Patterson HD Thompson R (1974) Maximum likelihood estimation of components of variance. Proc. Eighth Internat. Biom. Conf, 197-209

Rao CR, Kleffe J (1988) Estimation of variance components and applications. Amsterdam, The Netherlands

Rao PSRS (2001). Nonnegative estimators for the one-way random effects model. Commun Stat Theory Methods 30, 1605-1613

Rasch D, Verdooren LR, Gowers JI (1999) Fundamentals in the Design and Analysis of Experiments and Surveys. Oldenbourg Verlag, München, Germany

Rasch D, Masata O (2006) Methods of variance component estimation. Czech J Anim Sci 51, 227-235

Sahai H, Ojeda MM (2004) Analysis of variance for random models, Volume 1: Balanced Data. Birkhäuser, Boston, USA

Sahai H, Ojeda MM (2005) Analysis of variance for random models, Volume II: Unbalanced Data. Birkhäuser, Boston, USA

Searle SR (1971) Linear Models. Wiley, New York, USA

Searle SR, Casella G, McCulloch CE (1992) Variance Components. Wiley, New York, USA


Searle SR, Casella G, McCulloch CE (2006) Variance Components. Wiley, New York, USA

Shugart YY, O’Connell JR, Wilson AF (2002) An evaluation of the variance components approach: type I error, power and size of the estimated effect. Euro J Hum Genet 10, 133-136

Smith EJ, Savage TF (1992) A comparison of four methods of variance component estimation for heritability of embryonic mortality in turkeys. Poult Sci, 71, 229-234

Swallow WH, Monahan JF (1984) Monte-Carlo Comparison of ANOVA, MIVQUE, REML, and ML Estimators of Variance Components. Technometrics 26, 47-57

Thompson WA (1962) The problem of Negative Estimates of Variance Components. Ann Math Statist 33, 273-289

Verdooren LR (1982) How Large is the Probability for the Estimate of a Variance Component to be Negative? Biom J 24, 339-360

Westfall PH (1987) A comparison of variance component estimates for arbitrary underlying distributions. J Am Stat Assoc 82, 866-874

Westfall PH, Bremer RH (1994) Efficiency properties of cell means variance component estimates, J Stat Plan Inference 38, 159-178

Yolcu HI, Balcıoglu S, Frat MZ, Karabağ K (2004) Beyaz Yeni Zelanda Tavşanlarında canlı Ağırlıklara Ait Varyans Unsurlarının Farklı Yöntemlerle tahmini, Akdeniz Üniversitesi Ziraat Fakültesi Dergisi, 17, 81-85 [in Turkish]

Yang CC, Su CM (2005) [A Simulation Study on Estimators for G-coefficient of Generaliziability Theory]. J Educ Psych 28, 773-797 [in Chinese]

Received 7 July 2011, accepted 25 April 2012.

Corresponding author:

Mehmet Mendesemail: [email protected]

Canakkale Onsekiz Mart University, Terzioglu kampusu, Canakkale, Turkey

fraud detection system

Documents