designing field studies in soil science

7/28/2019 Designing Field Studies in Soil Science

1/10

1

Designing field studies in soil science

D. J. Pennock

Department of Soil Science, University of Saskatchewan, 51 Campus Drive, Saskatoon, Saskatchewan,Canada S7N 5A8 (e-mail: [email protected]). Received 22 May 2003, accepted 1 December 2003.

Pennock, D. J. 2004. Designing field studies in soil science. Can. J. Soil Sci. 84: 110. Field research in soil science ranges from modalprofile descriptions in support of soil survey to elaborate manipulative experimental designs. All of these field approaches make a valu-able contribution to soil science, but researchers who do not use either classical manipulative experimental or geostatistical designs havelittle guidance (or encouragement) available to them. Well-designed field research of any type requires a clear definition of the researchquestion; a thorough review of the literature to establish the state of knowledge; definition of the population under study and the elementsthat comprise it; and choice of appropriate scales for sampling support, spacing, and study extent based on an understanding of the under-lying processes. For studies where hypothesis testing is appropriate, the hypotheses should be based on sound biological or physical rea-soning, and sufficient replicates should be taken to ensure a reliable test. The major challenge in field research design is the developmentof landscape-scale research designs to examine complex interactions among hydrological, climatic, chemical, and biological processesat scales relevant for environmental management.

Key words: Research design, landscape-scale , soil genesis, pattern studies, hypothesis testing, spatial statistics, sampling, cesium

Pennock, D. J. 2004. La conception dtudes sur le terrain en science des sols . Can. J. Soil Sci. 84: 110. Dans la science dessols, la recherche sur le terrain va de la description modale de profils pour faciliter les levs en pdologie la conception dex-priences exigeant des manipulations complexes. Ces approches apportent une contribution utile cette science, mais leschercheurs qui ne font pas appel aux manipulations classiques ou aux analyses gostatistiques ont peu pour les guider (voire lesencourager). Une exprience sur le terrain bien conue ncessite une dfinition claire du but recherch, un dpouillement mtic-uleux de ce qui sest crit afin dtablir ltat des connaissances, la circonscription de la population ltude et des lments quila composent et, enfin, le choix dchelles dune grandeur convenable pour planifier plus aisment lchantillonnage, lespacementdes prlvements et lenvergure de lexprience en regard de nos connaissances sur les processus sous-jacents. Quand ltude partdune hypothse quil faut vrifier, cette dernire devrait sappuyer sur un solide raisonnement biologique ou physique, et le nom-bre de rptitions devrait suffire garantir la fiabilit de lexprience. La principale difficult lorsquon conoit une tude sur leterrain consiste dresser un plan exprimental lchelle du relief qui permettra ltude des interactions complexes entre lesprocessus hydrologiques, climatiques, chimiques et biologiques luvre, une chelle adapte la grance de lenvironnement.

Mots cls: Plan exprimental, chelle du relief, gense des sols, tudes de motifs, vrification dhypothse, statistiques spatiales,chantillonnage, csium

Statistical analysis and interpretation are the least critical aspects of experimentation, in that if purely statistical orinterpretative errors are made, the data can be reanalyzed. On the other hand, the only complete remedy for designor execution errors is repetition of the experiment.(Hurlbert 1984, p. 189)

Field research in soil science encompasses a great range ofactivities yet guidance on the design of field research programsis confined to only a few of these activities. Our colleagues whodeal with classical agronomic trials have research design andstatistical guidance available at every level from the simplecookbook to the most esoteric of explorations. In more recenttimes, pedometricians have established a range of appropriatefield-sampling designs for geostatistical and related spatial sta-tistical problems. Many of the rest of us, however, are some-where between these two well-charted research pathways.Often the objects we wish to study cannot be readily manipu-lated and imposed as treatments, which greatly complicates theapplication of classical agronomic research designs. The rig-orous lay-out and high sample number requirements for manypedometric sampling strategies can be daunting, and there aremany research questions these approaches are ill-suited toexplore. My purpose in this paper is to categorize the full range

of field studies undertaken by soil scientists and then to discussthe design of the field-sampling programs appropriate for eachof the categories. The main emphasis is placed on studies out-side of the classical agronomic and pedometric frameworks.

Although critical evaluation of field research studies hasnot been a major focus in soil science, it has engaged fieldecologists, hydrologists and physical geographers to a much

greater degree. The structure of this paper closely reflectsthe influential paper by Eberhardt and Thomas (1991) on thedesign of field studies in environmental science. They usetwo main criteria to classify field studies. The first criterionis the kind of events that are observed, which they divideinto those where a distinct perturbation occurs (e.g., a majorflood, toxic spill, or the establishment of a polluting indus-try in a region) and those where no distinct event occurs.

The second, and more important, criterion is the degree ofcontrol exerted by the observer. In a controlled experiment,


2/10

2 CANADIAN JOURNAL OF SOIL SCIENCE

treatments are imposed by the researcher to measure theresponse of the property under study. In uncontrolled stud-

ies, the treatments are typically a selection of classes orstrata that differ in some characteristic that is not under thecontrol of the researcherdifferent soil textural classes orlandform positions, for example. This distinction corre-sponds to the widely cited distinction between manipulativeand mensurative experiments made by Hurlbert (1984).

Classifying research studies is much like classifyingsoilsthere are a few textbook examples encountered andtypically the profiles/studies seem to fit into two or moreclasses. My intent here is to develop categories of studies

that have different sampling approaches associated with

them, rather than to attempt to classify individual research

papers into one category or another.

CATEGORIES OF FIELD RESEARCH IN SOILSCIENCE

An initial distinction can be drawn between studies whose

primary objective is the advancement of knowledge

through publication in peer-reviewed journals versus

research that is undertaken to meet specific public or pri-

vate sector objectives. In the latter case the research design

is often dictated (at least in part) by the initiating agency.

Table 1. Categories of field research in soil science

Initiator Category of study Sampling strategy Description

Private sector Contaminant Judgment Delineation of type and severity of contamination. Sampling strategysurvey may be dictated by appropriate regulatory requirement

Public sector Soil survey Judgment Mapping of soil taxonomic unit distribution accompanied by descriptive(free survey) summaries of modal soil profiles

Monitoring Judgment and Determination of current status and trends for natural resourcesprobability

Researcher Soil geomorphological/ Judgment Interpretation of soil or landform evolution based on description and analysispedological of high resolution soil stratigraphical sections

Pedometric/ Model-based Values of property modeled as a stochastic process; used to define spatialgeostatistical means, variances, and in interpolation of values at unsampled positions

Perturbation Design-based/ Examination of the effects of a distinct perturbation (e.g., flooding, clear-cutprobability forest harvest)

Pattern Design-based/ Hypothesis generation and testing based on analysis of spatial and temporalprobability patterns

Model support Judgment, model Studies undertaken to estimate parameters of physically based models, to develop

and design-based functional relationships between variables (e.g., pedotransfer functions) or to testsimulations produced by models

Comparative Design-based/ Hypothesis testing for differences and correlations among classes that cannotmensurative probability be randomized by the researcher (e.g., topographical position, soil textural class)

Manipulative Design-based/ Hypothesis testing based on comparisons among treatments that are imposed byprobabili ty the researcher (e.g., fertil izer rate and formulation)

Table 2. Number of samples required to achieve a power of 0.80 in a two-sample, two-tailed t-test. The effect size is expressed as the real difference(in percent) between the means of the two groups at a constant coefficient of variation of 20%

Effect size(% real differencebetween means) Probability of type I error ()

0.01 0.05 0.10 0.20Number of samples required for power of 0.80

5 376 253 199 14510 96 64 51 3715 44 29 23 1720 26 17 14 1025 17 12 9 730 13 9 7 540 8 6 4 350 6 4 3 3


3/10

PENNOCK DESIGNING FIELD STUDIES IN SOIL SCIENCE 3

The first three categories of research (contaminant survey,soil survey, and monitoring studies) are typically initiatedby private or public sector agencies (Table 1). The remain-der are researcher-initiated or curiosity driven projectsassuming, of course, that you can find a funding agencythat shares your curiosity!

(1) Contaminant SurveysThese types of field studies are most typically undertaken byprivate-sector environmental consultants, and the specificobjective may range from an initial evaluation of the extentof contamination to the final stage of remediation of theproblem. Laslett (1997) states that consultants who under-take these surveys almost always employ judgment sam-pling (also called purposive sampling). Typically theyprefer to place their samples where their experience andknowledge of site history tells them the contaminationmight be located. Judgment sampling can result in accurateestimates of population parameters such as means and totals,but cannot provide a measure of the accuracy of these para-meters (Gilbert 1987). In many jurisdictions the sampling

design may also be constrained by the appropriate regulato-ry framework.

(2) Soil SurveyThe objective of these studies is to map the distribution ofsoil taxonomic units and to provide descriptive summariesof the main properties of the soils (or, more simply, soil sur-vey). Basher (1997) observed that government-funded soilsurvey is in decline throughout much of the developedworld due to the combined effects of budgetary cuts and(often) the lack of an obvious user group for the products.Special-purpose inventories of soils are one of the coreactivities in soils-related environmental consulting.

In soil survey or inventory studies the association between

soil classes and landscape units is established in the field byjudicious selection of sampling points and thorough descrip-tion of the profiles (often termed the free survey approach).When exercised by a soil scientist whose experience allows awise selection and an informed interpretation of the profile,this type of judgment sampling can be an extremely efficientway of completing the inventory. In soil survey, judgment-based sampling is often complemented with probability sam-pling such as systematic grid or transect sampling in selectedunits as a quality control measure and to generate variates ofselected properties.

(3) Monitoring StudiesThe goal of these studies is to assess the current status andtrends of different natural resources. Olsen et al. (1999)surveyed the statistical issues surrounding the majornational inventory programs in the United States. Theyfound that a considerable range of sampling designs werebeing used and that generalization was difficult due to thewide range in objectives among the agencies that initiatedthe monitoring studies. Given the diversity of possibledesigns, they emphasized the need to have a clear and con-cise statement of monitoring objectives prior to selecting aspecific design.

(4) Pedological or Soil Geomorphic StudiesThe focus in these studies is on past eventsmore specificallyon the processes that formed the soil properties or landscapesunder study and the environments that controlled the rates ofthese processes. Two branches of this approach were evident inthe past pedon-scale studies and soil geomorphic or landscapeevolution studies. Pedon-scale studies were closely associated

with the development of soil taxonomic systems, and (like theclassification systems themselves) focus on vertical, intra-pedon processes. The roots of the soil-geomorphic approachare in Quaternary geology and soil science, and soil geomor-phologists focused on lateral transfer processes and historicallandscape evolution, the latter often through the tools of pale-opedology (Basher 1997).

The geomorphologist Stanley Schumm summarized theneed for historical reconstruction in the earth sciences in hisexamination of fluvial geomorphology:

Thus it is possible to view the fluvial system as aphysical system or a historical system. In actuality thefluvial system is a physical system with a history.

Hence the objective of the geomorphologist is tounderstand not only the physics and chemistry of thelandscape, but its alteration and evolution throughtime (Schumm 1977, p. 10).

Equally there are very few soil properties that can beexplained in isolation from their historical and landscape con-textA soil sample in the laboratory is nothing more than abag of dirt. That bag of dirt becomes a useful research sampleonly if we know the field relations it represents (Daniels 1988,p. 1518). Given the great interest in the effect of current andpredicted climate change on soils, it is unfortunate that theeffect of past climates on soil and landscape evolution is so lit-tle studied now by North American soil scientists.

In pedological or soil geomorphic studies, modal profiles orhighly resolved exposures are located, and their pedological orsedimentological attributes are carefully described and sam-pled. For the soil geomorphic studies, the stratigraphical extentof the formation is assessed and (ideally) relative or absolutedating techniques are used to develop a chronology of events.The field observations are combined with laboratory analysesof stable soil properties to develop an interpretation of the soil-forming processes. The oft-cited book by Birkeland (1999)provides a thorough examination of this approach, as does thereview by Wysocki et al. (2000). As Phillips (2001) points outin his article on contingency and generalization in pedology,there is rarely a single, definitive interpretation that emerges inthese studiesmultiple working hypotheses are not only intel-lectually desirable but may also all be true.

(5) Geostatistical/Pedometric StudiesThis category includes a wide range of approaches (Yatesand Warrick 2002; McBratney et al. 2002). Typically thesestudies are undertaken to quantify the spatial pattern of soilproperties, to use this quantification in the interpolation ofvalues at unsampled locations, to assess the suitability ofdifferent spatial models for processes, or in the design ofefficient sampling programs.


4/10


Brus and de Gruijter (1997) argued that there is a funda-mental distinction between geostatistical studies and thosebased on more classical statistical survey principles (includ-ing the remaining categories of studies defined in thispaper). Geostatistical studies use model-based samplingstrategies where the pedogenic process that has lead to thefield of values for a given property is modeled as a stochas-

tic process. The specific realization of the population that issampled is but one of an infinite set of populations thatcould be produced by the model. For model-based samplingstrategies randomness is inherent to the modelling process,and non-probability-based sampling designs can be accom-modated in the analysis.

In the alternative, design-based sampling strategies, thefield of values that we are to sample is assumed to be fixed(at least at the specific time of sampling) and stochasticity isdue solely to the probability based sampling design used andto any measurement error. The sampling design used con-fers a design-specific independence on the data regardless ofthe spatial continuity of the underlying processes. Brus andde Gruijter (1997) stated that the independence of sample

data required by classical statistical analysis is met by thisdesign-independence, which is contrary to information con-tained in many published sources.

(6) Perturbation StudiesEberhardt and Thomas (1991) categorized studies triggeredby a distinct perturbation as intervention or perturbationstudies. In ecology, perturbation studies often focus on theresponse of population characteristics to the perturbationusing time series analysis. Ideally, these studies are carriedout before and after the perturbation, and the post-perturba-tion results should ideally be compared to some control orunperturbated system (Eberhardt and Thomas 1991). Thesestudies are uncommon in the research literature in soil sci-

ence but are central to environmental pollution monitoring.Gilbert (1987) provides an excellent overview of samplingmethods in environmental pollution monitoring.

(7) Pattern StudiesThese studies are undertaken to assess and explain the spa-tial or temporal pattern of properties (Eberhardt andThomas 1991). Two major (and overlapping) themes existin pattern studiesthe quantification of the spatial andtemporal variability in properties and hypothesis generationand testing using point patterns (Underwood et al. 2000).The former are (typically) done with the explicit purpose ofevaluating variability; the latter are often much less clearlydesigned, and the hypotheses being tested are (too often)fuzzy and/or undefined.

Compared to the rich literature in the use of geostatistics/pedometrics there has been little attention paid by soil scien-tists to hypothesis generation and testing potential of patternstudies. Underwood et al. (2000) stated that pattern studiesoccupy a clear niche in the evolution of ecological research:

There is no possible doubt that observations of pat-terns or lack of patterns are the fundamental starting-blocks for ecological study. Until patterns have been

described, there is no basis for invoking explanatorymodels about processes. Nor is there any mechanismfor understanding the scale and scope of any processthat may be operating (p. 108)

In some cases, the hypothesis testing may involve formaltesting, but in many others the comparison between mea-

sured and hypothesized patterns may be made using othercriteria, including simple visual comparisons (Grayson andBlschl 2000).

This multi-faceted role of pattern studies is evident inmany areas of soil science. For example, Govers et al.(1999) pointed out that spatial pattern studies on soil erosionusing 137Cs redistribution techniques were critical in thediscovery of tillage redistribution of soil by soil scientistsand geomorphologists. Multiple studies in cultivated fieldsfound that consistently high rates of soil loss were associat-ed with slope segments that had convex downslope profiles.Soil loss from these segments could not be readily explainedby water or wind erosion and ultimately tillage redistribu-tion (which had been previously documented and explored

in the agricultural engineering literature) was recognized asthe most probable explanation for the observed pattern(Govers et al. 1999).

The nitrous oxide emission study of Wagner-Riddle et al.(1997) is an example of a temporal pattern study. They usedmicrometerological techniques and a Tunable Diode LaserTrace Gas Analyzer to generate a quasi-continuous recordof N

2O emissions from four fields that had different cropsover a 28-mo period. The crops are unreplicated, and there-fore unambiguous, causal linkage between the crops andemissions cannot be made; however, Groffman et al. (2000)argued these multi-year, quasi-continuous studies are essen-tial to challenge our existing ideas about the factors thatcontrol annual emissions.

Hypotheses testing using patterns in hydrology has recent-ly been explored in detail in the volume edited by Graysonand Blschl (2000). One of their main objectives is the com-parison of measured patterns of hydrological properties tosimulated patterns produced by process modeling. The meth-ods used for comparisons include purely visual comparisonsand point-by-point comparisons based on direct comparisonof points and mapping of residuals between observed andsimulated patterns. They suggest that more refined andinsightful use of spatial patterns is vital for progressing thescience of hydrology and for better constraining the uncer-tainty in our predictions of the future (p. 78).

Underwood et al. (2000) pointed out that pattern studies cansupply necessary support for a hypothesis but cannot providesufficient evidence for its acceptanceother, alternativeprocesses could often lead to the same prediction. This indeter-minacy of field observations is also emphasized by Phillips(2001). For example, the study of Wagner-Riddle (1997) clear-ly indicated the importance of freeze-thaw processes for annu-al N2O emissions but could not identify the specific mechanisminvolved. The pattern correlation study of Pennock et al. (1992)implies a causation linkage between properties, but uses a tech-nique (correlation analysis) which, by itself, cannot establishcausation (Webster, 1989). This reinforces the initial point of


5/10


Underwood et al. (2000) that these studies have a clear role toplay at an early stage of the development of a particular area ofstudy, but the hypotheses generated by them must be more rig-orously tested using other designs.

(8) Modelling Support StudiesEberhardt and Thomas (1991) discussed a distinct type of study(sampling for modelling) that is undertaken for efficient esti-mation of parameters for models. Their definition is a strict one,but this category is used in a more general way in my paper. In

soil science (and most particularly in soil physics and hydrolo-gy) many field studies are undertaken to support modelling: toestimate parameters required in physically based models, todevelop functional relations (Webster 1989) between variables,or to test simulations produced by models. The development ofpedotransfer functions (Bouma 1989) is a particular category ofthis type of study. The sampling designs used in these studiesrange from judgment selection of points through to very elabo-rate manipulative designs.

The final categories of experiments are all primarily con-cerned with the drawing of comparisons between groups. Theydiffer in the ability of the observer to impose the treatments,and correspond to the well-known mensurative/manipulativedistinction made by Hurlbert (1984).

(9) Comparative Mensurative StudiesIn comparative mensurative experiments, comparisons aremade between classes that the researcher defines but cannotcontrolsites grouped by different soil textures, soil zones,landform positions, soil taxonomic classes, and drainageclass are all examples. Their location cannot be randomizedby the researcher, unlike treatments such as tillage type orfertilizer rates.

Eberhardt and Thomas (1991) distinguish between stud-ies where the whole population is available for sampling andthose where a deliberate selection of contrasting parts of thepopulation is made. In soil science, this latter type wouldtypically be called indicator studies. Studies on landformand soil erosion relationships in Saskatchewan using 137Cscan be used to illustrate this distinction. Initial spatial pat-tern studies of 137Cs-estimated soil erosion suggested thatdistinct ranges of soil loss or gain were associated withquantitatively defined landform elements (Martz and de

Jong 1987; Pennock and de Jong 1987). Whole-populationstudies using sampling grids that spanned the range of land-forms at the sampled sites were used to test this associationbetween landform elements and soil redistribution (Pennockand de Jong 1990). These studies consistently showed thatdoubly convex slope segments [termed divergent shoulder(DSH) elements] had the highest rates of soil loss. Thisknowledge was used in Pennock et al. (1995) to comparemaximum erosion rates of five parent materials by onlysampling DSH elements. Their results provide a relativeranking or indication of the different parent materials sus-ceptibility to erosion, but only if the assumptions of thewhole-population studies are valid.

(10) Manipulative ExperimentsIn manipulative studies the treatments can be imposed bythe researcherideally as fixed amounts that are appliedexactly (or at least as exactly as the implementation allows).Eberhardt and Thomas (1991) argued that these are the onlytype of research designs that the term experiment shouldbe attached to. Any issue of the Canadian Journal of SoilScience will contain several such studies and many agricul-tural research design texts give excellent guidance for theselection and implementation of these methods. The main

Exten

t

Fig. 1. Schematic diagram illustrating the concepts of support, spacing, and extent for a grid sampling design on a near-level landscape[adapted from Blschl and Sivapalan (1995)].


6/10


issues facing soil scientists when selecting these techniquesare associated with replication. These issues are also com-mon to the comparative mensurative techniques and will bediscussed below.

CONSIDERATIONS IN THE DESIGN OF FIELDPROGRAMS

Selection of the Appropriate CategorySelection of the appropriate approach can be made onlyafter formulation of a clear research question. If myresearch question is a soil geomorphic one, the success ofthe project will hinge on my ability to find, describe, andconvincingly interpret soil stratigraphical sections thatyield a highly resolved environment record. Choice of anappropriate field design for other research questions willdepend in large part on the information already available.For a newly discovered property [e.g. amino sugar N(Khan et al. 2001) or charcoal in soil organic matter(Skjemstad et al. 2002)], inventory or pattern studies in anew region would be appropriate. On the other hand, it isdifficult to see what an inventory or single-site pattern

study would add to our understanding of soil nitrate ororganic carbon in almost any agricultural region in Canada.After a thorough review of the literature you may posequestions about these properties that an innovative manip-ulative design could successfully address; indeed the devel-opment of such designs is the hallmark of critical scienceand the making of great scientists. A thorough review ofwhat is known is essential to determine how to designresearch on what is (as yet) unknown.

Definition of Population and Experimental UnitsThe research question also defines the population that thestudys findings will pertain tosometimes referred to asthe inference space (at least when using inferential statis-

tics). A clear definition of the population is essential for suc-cessful sampling design.

The population consists of all possible objects that sharesome common characteristic; the sampling design or exper-imental design will specify how a subset of those objectswill be drawn from the population (Steel and Torrie 1980).In manipulative studies, the treatments are imposed on indi-vidual experimental units or experimental plots that can berandomly situated at the site; the size, shape and arrange-ment of the experimental units are decided upon by theresearchers (Steel and Torrie 1980).

For pattern studies and comparative mensurative studies,the population is composed of objects whose placement can-not be controlled by the researcher. Often the area that a sin-gle observation pertains to is much less obvious than in amanipulative study. The range of spatial dependence can bequantitatively assessed using geostatistics, and sufficientstudies have been completed to give the approximate rangefor many properties [see Mulla and McBratney (2000) for arecent summary]. The use of geostatistics prior to the designof a sampling program would be ideal but is a logistical andfunding obstacle for many researchers.

For many field research projects in soil science, hydrolo-gy, and plant ecology, the elements of the population are

topographically defined (Rowe 1984; Hudson 1992;Grayson and Blschl 2000). Grayson and Blschl (2000)discuss four main types of topographically defined ele-ments, which they generally call model elements. Manyapproaches to landform segmentation have been developed,which divide landforms into quantitatively defined landformelements (Speight 1968). In other studies, the population

may be composed of soil map polygons, textural groupings,or drainage classes. Based on the terminology of researcherssuch as Speight (1968) and Grayson and Blschl (2000), thegeneral termpopulation elements will be used in the remain-der of this paper for the objects that comprise the popula-tion. The term sample will pertain to the specific populationelements that are drawn from the population in a particularsampling design; the sampling design specifies how the spe-cific elements are selected (following well-establishedusage of these terms).

Scale IssuesScale refers to a characteristic length (or time) of a process,observation, or model (Blschl and Sivapalan 1995).

Grayson and Blschl (2000) examine three aspects of scale(Fig. 1). Supportis a geostatistical term and refers to the vol-ume of a spatial sample (e.g., core diameter) or the mea-surement length of a temporal sample. This is also referredto as sampling unit (Steel and Torrie 1980) or grain(McBratney 1998). Spacing pertains to the distance (orelapsed time) between successive sampling units and, moregenerally, the layout of the samples in the study area.

The extentis the total duration of the temporal series ofsamples or the length (or area) sampled in a spatial samplingprogram. Defined in this way, extent is the spatial or tem-poral definition of the population and follows directly fromthe definition of the population.

Issues of extent are relevant for the design of all field

studies. A common theme in the ecological (e.g., Carpenteret al. 1998; Gustafson 1998; Schindler 1998; Underwood etal. 2000; Groffman et al. 2000; Haag and Matschonat 2001)and hydrological (Seyfried and Wilcox 1995, Western et al.1999) literature is the need to match the extent of the studywith the scale of the process (or processes) under study.Seyfried and Wilcox (1995) provided several examples ofthe connection between extent and process. At one extreme,the processes of surface runoff and infiltration at one studysite were controlled by shrub spacing, and the relevant scalewas between 0.3 and 8 m. At the coarsest scale, variabilityin snowfall distribution at different elevations in a largecatchment was due to the effect of prevailing wind directionduring storms, and the relevant extent for this study wasbetween 2000 and 15 000 m. Hence, the appropriate extentof a study must be carefully matched to the processes thatthe researcher believes are operating in the area.

Replication IssuesQuestions about scale are closely linked to the issue of repli-cation in hypothesis-testing experiments. In a manipulativestudy, replication means the repeated imposition of a set oftreatments. In a mensurative or a pattern study, the repeated,unbiased selection and sampling of population elements


7/10


constitutes replication [albeit, as Eberhardt and Thomas(1991) point out, of a very different kind than in manipula-tive experiments]. Many classical agronomic experimentsinvolve both types of replication; for example, fertilizer tri-als at sites with different textures involve imposed treat-ments (fertilizer rates) and inherent treatments or sitecharacteristics (soil texture).

Replication is used to provide an estimate of experimen-tal error, to improve precision by reducing the standarddeviation of treatment or class means, to increase the scopeof inference, and to effect control of the error variance (Steeland Torrie 1980). The estimation of the experimental erroris required for tests of significance. In a standard, small plotexperimental design, lack of sufficient replication is inex-cusable and studies that failed to have sufficient replicationshould not be published.

For comparative mensurative, pattern or model supportstudies, the question of what constitutes a replicate is decid-ed by the research question and the definition of the popula-tion. For example, a study on the relationship betweenconvex profile curvature and rates of soil loss could be car-

ried out in a single field if the field contained multiple hillswith pronounced convex profile curvature (i.e., shoulderelements). The researcher could use intensive field samplingand gamma spectroscopy to measure the 137Cs concentra-tion in each of the shoulder elements, and then use conver-sion relationships to provide a time-integrated 40-yrmeasurement of soil loss due to erosional processes. Thecorrelation between soil loss and measured curvature at eachsampled point could then be assessed or a regression rela-tionship developed.

If, however, the researcher assumed that the results per-tained to all fields in the study region or to even larger unitssuch as soil zones, then they would be guilty of pseudorepli-cation (Hurlbert 1984). Pseudoreplication is a consequence

of the actual physical space over which samples are taken ormeasurements made being smaller or more restricted than theinference space implicit in the hypothesis being tested(Hurlbert 1984, p. 190). It occurs when researchers makeclaims that their work demonstrates a more general effectwhen, in fact, the error terms used in the statistical analysisreflects only the variation in a treatment or class replicate(Raffaelli and Moller 2000; Underwood 1997). The 137Csstudy discussed above provides a time-integrated measure ofthe soil redistribution pattern in the sampled field, but eachfield in the region has its own history of erosionfor exam-ple, catastrophic rainfall events may affect adjacent fieldsvery differently, depending on the crop or the residue coveron the surface during the event. Each field constitutes one ele-ment of the population of possible erosional responses; sam-pling of multiple fields (i.e., population elements) will allowthe parameters of that population to be estimated. Multiplecores from one field are a sub-sample of one field with a par-ticular erosional history; assuming each of these sub-samplesare independent replicates of the range of possible erosionalhistories is pseudoreplication. Clearly the way to avoidpseudoreplication is to have a clear definition of the popula-tion your study will pertain to and the elements that comprisethe population.

Hurlberts (1984) definition of pseudoreplication alsoincluded a requirement for replicates to be spatially inde-pendent. This requirement for spatial independence hasbeen challenged by subsequent authors (Underwood 1997;Raffaelli and Moller 2000). They argue that spatially non-independent replicates can be analyzed if the non-indepen-dence is explicitly incorporated into the analysis used, and

Underwoods (1997) textbook deals at length with suitablemethods. Moreover Brus and de Gruijter (1997) stated thatthe need for spatial independence has been misunderstoodthe design- independence inherent to specific probability-based sampling designs confers the required independenceon the samples.

A final, but very important, issue concerning replication isthat there are some things that cannot be replicated. An obviouscategory is catastrophesfloods, pestilence, or major pollutionevents that will (one hopes) not be repeated. Another categoryis the post-establishment effects of major projects such assmelters or dams. The two cases differ insofar as the occurrenceof the catastrophes cannot be foreseen, whereas the establish-ment of a power plant or dam could be. Both of these are types

of perturbation analyses, and Eberhardt and Thomas (1991)suggest that comparison of the affected area with well-selectedcontrol areas (i.e., unaffected by the perturbation) is a viableapproach in these types of studies.

There are also objects that cannot be replicated by virtueof their size or complexity. Schindler (1998) and Carpenteret al. (1998) argued that true replication is impossibleamong even small lakes such as in the Experimental LakesArea of Northwestern Ontario. These authors make a strongcase that we should not attempt to replicate systems at thislevel of complexity, but should instead have well-designedmanipulations of contrasting systems and then analyze themusing appropriate statistical techniques. Schindler (1998) isespecially critical of the use of replicated microcosm as sur-

rogates for whole-ecosystem manipulations because of theinability of microcosms to include ecosystem-scale transferand exchange processes. He concludes that proper upscalingfrom microcosm-type experiment is impossible, and experi-ments at less than ecosystem scales are inappropriate formaking ecosystem-scale predictions.

In soil science the term landscape-scale studies hasbeen commonly used instead of ecosystem-scale studies.True landscape-scale studies require inclusion of land useeffects, yet the extent of the landscape clearly limits ourability to truly replicate them and to impose treatments onthem. Fortunately the recent increase in publications by soilscientists (e.g., Van Kessel and Wendroth 2001) in this areaseems to indicate a desire to grapple with these issues.

Once the researcher has clearly defined what constitutes areplicate they are immediately confronted with the questionof how many replicates are required. In the simplest type ofhypothesis testing, two hypotheses are constructed: the nullhypothesis (Ho) of no difference between the two groups,and the alternative hypothesis of a significant differenceoccurring. The researcher chooses an level to control theprobability of rejecting the null hypothesis when it is actu-ally true (i.e., of finding a difference between the two groupswhen none, in fact, existed in nature or a Type I error).


8/10


For conservation research, Peterman (1990) argued that theconsequences of committing a Type II error (i.e., of failing toreject the null hypothesis when it is, in fact, false) can begraver than a Type I error. The probability of failing to rejectthe Ho when it is, in fact, false is designated as and thepower of a test equals (1). Low power tests of hypothesesare unlikely to detect differences between two or more groups

even when a difference does, in fact, exist in nature. Althoughthe appropriate power level for a reliable test must reflect thegravity of the issue under examination, Peterman (1990) sug-gested that, in general, studies should be designed to achievea minimum of 0.2 and hence a power of 0.80. Issues aboutthe selection of for environmental pollution studies areexplored in detail by Gilbert (1987).

The power of a test is a function of four factors: the level chosen, the variance of the sampled groups, samplenumbers (N) in each group, and the effect size (i.e., the actu-al difference between the groups). The smaller the probabil-ity of committing a Type I error (), the lower the power(holding the other factors constant). Peterman (1990) ques-tioned the uncritical acceptance of an of 0.05 or 0.01 for

conservation work, and argued that an of 0.10 or greateris more appropriate for some designs. Second, the higher thevariance in the groups, the lower the power. Soil propertiestend to be inherently variable and coefficients of variation(CV) range from about 10% for pH and porosity to 20% forSOC and texture and greater than 100% for many water orsolute transport properties (Mulla and McBratney 2000).This inherent variation must be taken into account in theselection of appropriate sample numbers. Power alsodecreases when the difference between the means of thesampled groups is small, or when the sample size is small.

An assessment of effects of the four factors that controlpower can (and should) be a required part of sampling design.For example, a researcher wants to determine the effects of

adoption of no-till on soil organic carbon. Two fields arebeing compared, and the researcher wants to know how manysamples must be taken to detect a statistical difference at an of 0.10 using a two-sample, two-tailed t-test. She believes thatthe two fields had SOC levels of about 50 Mg ha1 in theupper 15 cm before imposition of no-till on one of the fields10 years previously. She also assumes that the CV for SOC isabout 20%. Published summaries of SOC change due to no-till adoption in the Canadian prairies suggest that SOC gainsmay be as high as 5 Mg ha1 over the 10-year period (Janzenet al. 1998), yielding an effect size of approximately 10%. Apower analysis indicates that she requires 51 samples fromeach group to detect a difference for this effect size at her cho-sen of 0.10 (Table 1). If she had used only 25 samples pergroup, the P value of the t-test would equal about 0.24 and shewould fail to reject the null hypothesis. To detect smallchanges in SOC content with time, the use of well-structuredsamplings of the same sampling unit can allow much smallerdifferences to be detected over short periods of time (Ellertet al. 2002), and these specialized designs should be consid-ered for this research question.

This example illustrates the well-deserved skepticism thatmany statisticians have concerning significance testing ofdifferences. Webster (2001) pointed out that if you take

large enough samples you can establish that any soil is dif-ferent from almost any other for whatever property of themthat you care to choose (p. 335) and that rejection of thenull hypothesis does not mean that the difference you havedetected is important or physically or biologically meaning-ful. The researcher should initially establish the differencesamong groups that would have real significance from a

biological, physical, or (for many agronomic or environ-mental issues) economic perspective. Given what is knownof the inherent variation of the property under study, an ade-quate number of samples can be then taken to ensure reli-able difference testing of the hypotheses.

Sample Spacing and LayoutSeveral recent, comprehensive reviews have presented theoptions available for sampling design (Mulla andMcBratney 2000; Pennock and Appleby 2002; de Gruijter2003). Systematic sampling designs are commonly used inthe soil and earth sciences. They are often criticized by sta-tisticians [for reasons discussed in Eberhardt and Thomas(1991)] but the ease with which they can be used and the

efficiency with which they gather information make thempopular in the earth sciences. For example, Wolcott andChurch (1991) found that for sampling of river gravels up to500 randomly chosen points were required in order to yieldthe same quality of statistical information as 100 systematicgrid samples. The major caution in the use of systematicsampling with a constant spacing is that the objects to besampled must not be arranged in an orderly manner, whichmight correspond to the spacing along a transect or grid.

Geostatistical studies ideally require observations to bemade at two or more scales such that the spatial dependencecan be evaluated at different spacings (McBratney 1998).For example, Lettner et al. (2000) used three grids withareas of 10 000 m2, 100 m2, and 1 m2, each of which con-

tained 81 sampling points for a study on the spatial variabil-ity of137Cs. The smaller area grids were nested within thelarger grid, such that 235 samples were taken in all.

The use of non-stratified, systematic designs may be veryinefficient for mensurative sampling designs. Appropriatesample numbers for mensurative experiments can be gath-ered efficiently by an a priori placement of points into therelevant groups or strata, and then a random selection ofpoints within each stratum until the desired number isreached. For example, Slobodian et al. (2002) laid out a gridon the surface of their sites and then classified each point inthe grid into one of three landform element classes. Gridpoints were then randomly selected until 10 samples in eachof the three groups were chosen. The summary statistics forthe site as a whole can then be calculated by using design-specific estimators (de Gruijter 2002).

CONCLUSIONS AND RECOMMENDATIONSThe issues that should be considered in the design of a suc-cessful field research program can be summarized as:

1. A clear definition of the research question is the initial(and most critical) step. This definition dictates the type ofresearch design that is appropriate and the specific designissues associated with different research types.


9/10


2. The appropriateness of a given research design can bejudged only after a thorough review of what is known aboutthe research question. Exploratory pattern studies can bevery informative at an early stage of research, but yield lit-tle new information for well-established research topics.Equally, the imposition of a set of treatments if little isknown of the processes controlling responses is unlikely to

produce comprehensive interpretations.

3. There is never a good reason for haphazard samplingthe rationale for selecting sampling points in pedological,soil geomorphic, or inventory studies should be clearly stat-ed.

4. A clear definition of the population and the elementsthat comprise the population under study is very important.

5. The definition of the population dictates the extent ofthe study and the physical or temporal space that the resultspertain to, which is critical to avoid pseudoreplication.

6. The sample support, spacing, and extent of the studymust be consistent with what is known of the processes con-trolling the phenomena being studied.

7. The construction of hypotheses for formal testingshould be based on sound physical or biological reasoning,and sufficient samples should be taken to allow reliable test-ing of the alternative hypotheses.

8. The exclusion of phenomena because they cannot bereplicated is inherently limiting to the expansion of ourknowledge of soils. Innovative approaches must continue tobe developed and applied so that we can expand the scale atwhich field studies can undertaken.

This final point is a key one for the future of field researchin soil science. The recognition of the importance of tillagetranslocation of soil can again illustrate this point. Erosionhad been studied for decades using a well-established, wide-ly disseminated manipulative research designthe standarderosion plots used to establish and revise the Universal SoilLoss Equation. These plots had yielded a vast store of knowl-edge on the relationship between water erosion and site fac-tors such as slope gradient, management, and soil factors. Yetthe standard design of the experimental plotsrectangularplots located on rectilinear slope segmentsexcluded ele-ments with downslope curvature where soil movement bytillage translocation is most evident (Govers et al. 1999).Recognition of the importance of tillage translocation couldonly occur when innovative field sampling techniques werecoupled with 137Cs analysis of soils to yield our currentunderstanding of the spatial pattern of redistribution.

Progress in many of areas of soil science awaits similarinnovations in field research design. A major limitation toinnovation (and source of frustration for field researchers) isthe uncritical application by reviewers or editors of researchstandards suitable for small-plot, manipulative studies tomore complex field settingsfor example, demands for repli-

cation of unreplicable, complex landscapes or for randomiza-tion of fixed landscape elements. If the authors of a manu-script have clearly documented their rationale for the designthey selected (following the criteria presented at the begin-ning of this section) and have developed a coherent interpre-tation of the data gathered in the study, the research should bepublished. Subsequent testing of the interpretation by other

researchers using ever more rigorous designs is the best mea-sure of the ultimate importance of the research.

ACKNOWLEDGMENTSMy understanding of field research has been greatly improvedby discussions with colleagues past and present, particularlyDarwin Anderson, Chris Van Kessel, Richard Farrell, and TomVeldkamp. My thanks also to the three reviewers of this paper,who greatly improved the manuscript and suggested a numberof fine literature sources for citation.

Basher, L. R. 1997. Is pedology dead and buried? Aust. J. SoilRes. 35: 979994.Birkeland, P. W. 1999. Soils and geomorphology. Oxford

University Press, Oxford, UK. 430 pp.Blschl, G. and Sivapalan, M. 1995. Scale issues in hydrologicalmodelling: A review. Hydro. Proc. 9: 251290.Bouma, J. 1989. Land qualities in space and time. Pages 313 inJ. Bouma and A. K. Bregt, eds. Land qualities in space and time.PUDOC, Wageningen, The Netherlands.Brus, D. J. and de Gruijter, J. J. 1997. Random sampling or geo-statistical modelling? Choosing between design-based and model-based sampling strategies for soil (with discussion). Geoderma 80:144.Carpenter, S. R., Cole, J. J., Essington, T. E., Hodgson, J. R.,Houser, J. N., Kitchell, J. F. and Pace, M. L. 1998. Evaluatingalternative explanations in ecosystem experiments. Ecosystems 1:335344.Daniels, R. B. 1988. Pedology, a field or laboratory science? Soil

Sci. Soc. Am. J. 52: 15181519.de Gruijter, J. J. 2002. Sampling. Pages 4579 in J. H. Dane andG. C. Topp, eds. Methods of soil analysis. Part 4. Physical meth-ods. SSSA, Inc., Madison, WI.Eberhardt, L. L. and Thomas, J. M. 1991. Designing environ-mental field studies. Ecol. Monogr. 61: 5373.Ellert, B. H., Janzen, H. H. and Entz, T. 2002. Assessment of amethod to measure temporal change in soil carbon storage. SoilSci. Soc. Am. J. 66: 16871695.Gilbert, R. O. 1987. Statistical methods for environmental pollu-tion monitoring. Van Nostrand Reinhold, New York, NY. 320 pp.Govers, G., Lobb, D. A. and Quine, T. A. 1999. Tillage erosionand translocation: emergence of a new paradigm in soil erosionresearch. Soil Tillage Res. 51: 167174.Grayson, R. B. and Blschl, G. 2000. Spatial patterns in catch-ment hydrology. Observations and modelling. CambridgeUniversity Press, Cambridge, UK. 404 pp.Groffman, P. M., Brumme, R., Butterbach-Bahl, K., Dobbie, K.E., Mosier, A. R., Ojima, D. S., Papen, H., Parton, W. J., Smith,K. A. and Wagner-Riddle, C. 2000. Evaluating annual nitrous oxidefluxes at the ecosystem scale. Global Biogeo. Cycles 14: 10611070.Gustafson, E. J. 1998. Quantifying landscape spatial pattern:What is the state of the art? Ecosystems 1: 143156.Haag, D. and Matschonat, G. 2001. Limitations of controlledexperimental systems as models for natural systems: a conceptualassessment of experimental practices in biogeochemistry and soilscience. Sci. Total Environ. 277: 199216.


10/10


Hudson, B. D. 1992. The soil survey as paradigm-based science.Soil Sci. Soc. Am. J. 56: 836841.Hurlbert, S. H. 1984. Pseudoreplication and the design of ecolog-ical field experiments. Ecol. Monogr. 54: 187211.Janzen, H. H., Campbell, C. A., Izaurralde, R. C., Ellert, B. H.,Juma, N., McGill, W. B. and Zentner, R. P. 1998. Managementeffects on soil C storage on the Canadian prairies. Soil Tillage Res. 47:181195.

Khan, S. A., Mulvaney, R. L. and Hoeft, R. G. 2001. A simpletest for detecting sites that are nonresponsive to nitrogen fertiliza-tion. Soil Sci. Soc. Am. J. 65: 17511760.Laslett, G. M. 1997. Discussion of : D. J. Brus and J. J. de Gruijter,Random sampling or geostatistical modelling. Geoderma 80: 4549.Lettner, H., Bossew, P. and Hubmer, A. K. 2000. Spatial vari-ability of fallout Caesium-137 in Austrian Alpine regions. J.Environ. Radioact. 47: 7182.Martz, L. W. and de Jong, E. 1987. Using cesium-137 to assessthe variability of net soil erosion and its association with topogra-phy in a Canadian Prairie landscape. Catena 14: 439451.McBratney, A. B. 1998. Some considerations on methods for spa-tially aggregating and disaggregating soil information. Nutr. Cycl.Agroecosyst. 50: 5162.McBratney, A. B., Anderson, A. N., Lark, R. M. and Odeh, I.

O. 2002. Newer application techniques. Pages 159198 in J. H.Dane and G. C. Topp, eds. Methods of soil analysis. Part 4.Physical methods. SSSA, Inc., Madison, WI.Mulla, D. J. and McBratney, A. B. 2000. Soil spatial variability.Pages A321A352 in M. E. Sumner, ed. Handbook of soil science.CRC Press, Boca Raton, FL.Olsen, A. R., Sedransk, J., Edwards, D., Gotway, C. A., Liggett,W., Rathbun, S., Reckhow, K. H. and Young, L. J. 1999.Statistical issues for monitoring ecological and natural resources inthe United States. Environ. Monitor. Assess. 54: 145.Pennock, D. J. and Appleby, P. G.2002. Site selection and samplingdesign. Pages 1540 in F. Zapata, ed. Handbook for the assessment ofsoil erosion and sedimentation using environmental radionuclides.Kluwer Academic Publishers, Amsterdam, The Netherlands.Pennock, D. J. and de Jong, E. 1987. The influence of slope cur-

vature on soil erosion and deposition in hummocky terrain. SoilSci. 144: 209218.Pennock, D. J. and de Jong, E. 1990. Rates of soil redistributionassociated with soil zones and slope classes in SouthernSaskatchewan. Can. J. Soil Sci. 70: 325334.Pennock, D. J., Lemmen, D. S. and de Jong, E. 1995. Cesium-137-measured erosion rates for soils of five parent -material groupsin southwestern Saskatchewan. Can. J. Soil Sci. 75: 205210.Pennock, D. J., van Kessel, C., Farrell, R. E. and Sutherland,R. A. 1992. Landscape-scale variations in denitrification. Soil Sci.Soc. Am. J. 56: 770776.Peterman, R. M. 1990. Statistical power analysis can improve fish-eries research and management. Can. J. Fish Aquat. Sci. 47: 215.Phillips, J. D. 2001. Contingency and generalization in pedology,as exemplified by texture-contrast soils. Geoderma 102: 347370.Raffaelli, D. and Moller, H. 2000. Manipulative field experimentsin animal ecology: Do they promise more than they can deliver?Pages 299338 in A. H. Fitter and D. G. Raffaelli, eds. Advances inecological research. Volume 30. Academic Press, San Diego, CA.

Rowe, J. S. 1984. Forestland classification: limitations of the useof vegetation. Pages 132148 in J. G. Bockheim, ed. Forest landclassification: Experiences, problems, perspectives. Department ofSoil Science, University of Wisconsin, Madison, WI.Schindler, D. W. 1998. Replication versus realism: The need forecosystem-scale experiments. Ecosystems 1: 323334.Schumm, S. A. 1977. The fluvial system. John Wiley and Sons,New York, NY. 338.

Seyfried, M. S. and Wilcox, B. P. 1995. Scale and the nature ofspatial variability: Field examples having implications for hydro-logic modeling. Water Res. Res. 31: 173184.Skjemstad, J. O., Reicosky, D. C., Wilts, A. R. and McGowan,J. A. 2002. Charcoal carbon in U. S. agricultural soils. Soil Sci.Soc. Am. J. 66: 12491255.Slobodian, N., Van Rees, K. and Pennock, D. J. 2002.Cultivation-induced effects on belowground biomass and organiccarbon. Soil Sci. Soc. Am. J. 66: 924930.Speight, J. G. 1968. Parametric description of landform. Page239250 in G. A. Stewart, ed. Land evaluation. MacMillan ofAustralia, Sydney, Australia.Steel, R. G. D. and Torrie, J. H. 1980. Principles and proceduresof statistics. A biometrical approach. McGraw-Hill BookCompany, New York, NY. 633 pp.

Underwood, A. J. 1997. Experiments in ecology. Their logicaldesign and interpretation using analysis of variance. CambridgeUniversity Press, Cambridge, UK. 504 pp.Underwood, A. J., Chapman, M. G. and Connell, S. D. 2000.Observations in ecology: you cant make progress on processeswithout understanding the patterns. J. Exp. Marine Biol. Ecol. 250:97115.Van Kessel, C. and Wendroth, O. 2001. Landscape researchexploring ecosystem processes and their relationship at differentscales in space and time. Soil Tillage Res. 58: 9798.Wagner-Riddle, C., Thurtell, G. W., Kidd, G. K., Beauchamp,E. G. and Sweetman, R. 1997. Estimates of nitrous oxide emis-sions from agricultural fields over 28 months. Can. J. Soil Sci. 77:135144.Webster, R. 1989. Is regression what you really want? Soil Use

Manage. 5: 4753.Webster, R. 2001. Statistics to support soil research and their pre-sentation. Eur. J. Soil Sci. 52: 331340.Western, A. W., Grayson, R. B., Blschl, G., Willgoose, G. R.and McMahon, T. A. 1999. Observed spatial organization of soilmoisture and its relation to terrain indices. Water Res. Res. 35:797810.Wolcott, J. and Church, M. 1991. Strategies for sampling spa-tially heterogeneous phenomena: The example of river gravels. J.Sediment. Petrol. 61: 534543.Wysocki, D. A., Schoeneberger, P. J. and LaGarry, H. E. 2000.Geomorphology of soil landscapes. Pages E-5 to E-39 in M. E.Sumner, ed. Handbook of soil science. CRC Press, Boca Raton, FL.Yates, S. R. and Warrick, A. W. 2002. Geostatistics. Pages81118 in J. H. Dane and G. C. Topp, eds. Methods of soil analy-sis. Part 4. Physical methods. SSSA, Inc., Madison, WI.

designing field studies in soil science

Documents