jmp-incomplete block design

Chapter 7

Incomplete block designs

Contents7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5497.2 Example: Investigate differences in water quality . . . . . . . . . . . . . . . . . . . . 550

7.1 Introduction

Blocking (or stratification or pairing) is a fundamental aspect of statistics. The idea behind blocking is tocontrol for sources of variation that make it difficult to detect differences in the mean response betweentreatments. For example, in the stream slope example, the density of fish will vary among streams due tointrinsic differences among streams (e.g. different productivity) that is difficult to measure. Consequently,a good experiment will examine all treatments (e.g. the different stream slopes) in all streams so that theintrinsic differences in density among streams will “cancel out” when comparisons among treatments aremade.

In the examples in the earlier chapter, blocks were complete in that every block had every treatmentoccurring at least once and every treatment occurred in every block at least once.

In some cases, blocks are incomplete, either by design (e.g. blocks are too small) or by accident (e.g.missing values). Some care needs to be taken in the analysis of incomplete block designs, but a basic analysisis straightforward with modern software.

Please refer to the earlier chapter on the analysis of complete block designs as the points made thereabout examining assumptions are equally pertinent here.

549

CHAPTER 7. INCOMPLETE BLOCK DESIGNS

7.2 Example: Investigate differences in water quality

Water quality monitoring studies often take the form of incomplete block designs. For example, the fol-lowing data represents TSS in water samples taken upstream of a development (the reference sample), atthe development (the mid-stream sample), or downstream of the development (the ds sample). Samples aretaken during storm events when water quality may be compromised by the development. Here is a small setof data1:

Location Storm 1 Storm 2 Storm 3 Storm 4

Ref · · 25 20

Mid 51 · 100 ·DS 173 137 170 110

The · represents data that is missing. We will assume that the missing data are MCAR (Missing Completelyat Random), i.e. that missingness is unrelated to the value of TSS or any other measurable covariate in thestudy. One way this could be violated is if missing value indicate that the TSS reading exceed the toleranceof the measurement device.

In many cases, some of the data may also be censored, i.e. < LDL or > UDL where LDL and UDL arethe lower and upper detection limits. If censored data are present, more advanced methods are available.

Water quality varies among the storm events in some unknown fashion, but it is though that all locationsshould be influenced in the same way. For example, events with large amounts of precipitation may increasethe TSS in all locations.

How should such data be analyzed? Looking at the raw data, it appears that water quality levels at theDS site are about three times that at the Mid site; and in turn, the water quality at the mid site is about twicethat of the Ref site. However, a comparison of the simple average of the values in each location is an unfaircomparison because not all locations were measured on all storm events and the different averages wouldcompare different combinations of storm events.

An incomplete-block analysis takes into account the pattern of missing values. For example, if you wishto look at the ref vs. mid locations should use the data from Storm 3; the comparison of ref and ds locationsshould use the storm 4 event; and the comparison of the mid and ds locations should look at the storm 1 and3 events.

The dataset is available in the water-quality.jmp data file available in the Sample Program Library athttp://www.stat.sfu.ca/~cschwarz/Stat-650/Notes/MyPrograms.

We start by entering the data into JMP in the usual way:

1Such a small set of data likely has very poor power to detect anything but very large differences in water quality among the threelocations. Before conducting such a study, please perform a power analysis to ensure that sufficient samples are taken.

c©2012 Carl James Schwarz 550

http://www.stat.sfu.ca/~cschwarz/Stat-650/Notes/MyPrograms


In many cases, where data has a wide range and where the ratio among values is of interest, a log-transformation of the data is often preferred. A formula is used to create a column of the log-values (thelog-function is under the transcendental option of the function groups). Note that the log function is thenatural logarithm (i.e. to base e) and not the common (i.e. to base 10) logarithm.



This gives the revised data table:



The shorthand notation for the model is:

logTSS = Event Location

This model syntax is interpreted as saying that variation in readings of logTSS may be attributable to effectsdue to different location and to different storm events.

A key assumption being made in analysis of data collected under a blocked design is that the relationshipamong the treatments is the same among all blocks (i.e. no block-treatment interaction). In this case, thisassumption takes place at the log-level. If it is believed that the relationship among treatments differs amongblocks, then there is no simple way to analyze this experiment. Because of the importance of this assump-tion, it is recommended that any data collection should follow a generalized-block design where replicateobservations at some of the treatments takes place in each block2

This model is fit in JMP using the Analyze->Fit Model platform:

2Refer to Addelman, S. (1969). The Generalized Randomized Block Design. American Statistician, 23, 35-36. http://dx.doi.org/10.2307/2681737. and Gates C. E. (1999). What really is experimental error in block designs. American Statistician49, 362-363. http://dx.doi.org/10.2307/2684574


http://dx.doi.org/10.2307/2681737

http://dx.doi.org/10.2307/2681737

http://dx.doi.org/10.2307/2684574


Notice that Location and Event must be nominal or ordinal scaled variables.

The hypothesis of interest is that:

H :µlocation ds = µlocation mid = µlocation ref

A :not the above

where the µ terms refer to the mean logTSS at each of the three locations.

In this first analysis, blocks are treated as fixed effects. This is known as the Intra-block analysis. Theprograms automatically account for the missing values – note that in some cases, the design is known asnon-connected, and the analysis can fail. This typically happens with extreme numbers of missing values –contact me for more details.

We start by looking at the F -test for location effects. The Effect Test is:


SUN

Highlight


The F -statistic is about 25 with a p-value of .0382. As this is somewhat smaller than α = .05, there is someevidence of a difference in the mean logTSS among the three locations.

It is instructive to look at the estimated differences among the mean logTSS: in the different locations.


SUN

Highlight


As in previous chapters, a multiple-comparison procedure (the Tukey HSD procedure) should be used tocontrol the experimentwise error rate. Please consult earlier chapters for details.

The estimated difference in the mean logTSS between the ds and ref locations is 1.91 (SE .27). Thisimplies that the TSS at the ds location is estimated to be e1.91 = 6.75 TIMES larger (on average) than at theref site. The se for the ratio is NOT found by simply taking the anti-log of the se on the log-scale. However,by application of a technique called the delta-method, it is possible to show that the se of the anti-log of theestimate is found as

SEanti−log = SElog(6.75) = 1.82

By taking the anti-logarithms of the confidence interval for the difference in mean logTSS, we find that weare 95% confidence that the ratio of TSS between the ds and ref sites is (e.31 = 1.36→ e3.51 = 33.4) timeslarger. Notice that while the confidence interval is symmetric on the log-scale, it is not symmetric on theanti-log scale.

The confidence intervals are much wider than the usual±2se because the total sample size is only 8, butthere are 6 parameters that are estimated leaving only 2 df for the residual error. The confidence interval



multiplier with 2 df is considerably larger than the multiplier of 1.96 (or about 2) used when sample sizesare large.

Note that the estimated differences above automatically adjust for the missing values and are NOT equalto the differences in the raw mean (see below).

The average logTSS across storm events can also be found.:

Notice that the estimated “LSmeans” or "population means" is different than the raw mean. This is becauseof the adjustment by the procedure for the pattern of missing values. The precision of each marginal meandiffers because of the differing amount of samples collected at each location. The anti-logarithm of eachmarginal mean would be interpreted as an estimate of the geometric mean TSS at each location.

Of course, the other assumptions made for any ANOVA need to be checked (i.e. equal variance amongtreatment groups; independence of residual; no outliers; normality of residuals; X measured without error,etc.) as in previous chapters. Don’t forget to look at the residual plots. Unfortunately, with such limiteddata, there is likely to be very little power to detect anything but gross violations of the assumptions.



Final Comment: A more refined analysis would treat the storm events as random effects. The analysiswould proceed as above except Events are declared as a random effect.

Notice how the random “block” is specified in the Analyze->Fit Model dialogue box:



Ironically, in cases where blocks are incomplete, there are two sources of information about treatmenteffects. The major part of the information comes from the intra-block analysis (done above). Some smallamount of additional information can be extracted (known as the inter-block information). By specifyingthat blocks are a random effect (i.e. that you wish to extrapolate to other events other than the observedstorms), it is possible to combine both analyses with modern software.

The hypothesis of interest is the same under the fixed and random block models.

The effect test uses more information and so indicates more evidence of an effect of location upon themean logTSS:



Notice how the F -statistic and p-value changes slightly for the test of no location effects compared to theintra-block analysis (where blocks are fixed effects). This revised model extracts additional information (theinter-block information) from the data that the first model ignored.

The estimated DIFFERENCE of the mean logTSS are slightly changed (no dramatic change) but thestandard error is improved3:

3If there were many blocks, the standard error of the combined inter- and intra-block analysis could be dramatically improved



Similarly, there is little change in the estimated marginal means (over all storm events) but some im-provement in the estimated standard errors.



With modern software, the analysis of incomplete block designs is fairly straightforward. In some casesyou can run into problems if there are substantial missing data in a systematic pattern (the design is notconnected). Please consult a statistician for details on such models.


jmp-incomplete block design

Documents