Key Words

ssttaattiissttiiccss;; qquuaalliittyy aassssuurraannccee;; ssaammpplliinngg

Introduction

The concept of sampling is straightforward,and it is generally accepted today that

selecting a correct sample is certainly advan-tageous to the experimenter, who must analyse the data and finally summarise the results.Exactly why sampling needs to be performed, andat what financial cost, remain, however, issuesthat need to be addressed. The second of thesetwo issues cannot be easily answered, as the finan-cial costs will vary according to type, duration,and importance of the experiment. The first, how-ever, can be addressed. Likewise, the approachesthat one can employ in order to select the correctsample can also be identified. This paper will lookat the need for sampling and will justify why it is sometimes sensible to select a sample according to a few fundamental rules or conditions. Further, it will focus on sampling in the field ofquality assurance and will compare the basic sta-tistical concepts to international standards. Thedetailed methods that can be employed to selectan appropriate sample are not specificallyaddressed in this paper, as detailed discussionwould be unwieldy for such an article. Theyshould, however, remain a topic for possiblefuture articles.

Why Should we Sample?

Why should we take a sample of anything? Thereare many reasons for not taking any sample; the

Statistical Sampling — Practical Advicein the Field of Quality Assurance

Martin Scott*

Roche Diagnostics GmbH, Pharma Development Biometrics, Sandhoferstrasse 176, 68305 Mannheim, Germany

Copyright © 2001 John Wiley & Sons, Ltd. Qual Assur J 2001; 55, 157–161.DOI: 10.1002/qaj.146

* Correspondence to: M. Scott, Roche Diagnostics GmbH, PharmaDevelopment Biometrics, Sandhoferstrasse 176, 68305 Mannheim,Germany.E-mail: martin.scott@roche.com

Summary

AAlltthhoouugghh mmaannyy ppeeooppllee wwhhoo ccoonndduucctt eexxppeerrii-mmeennttss aacccceepptt tthhaatt iinn cceerrttaaiinn sscceennaarriiooss ssoommeessoorrtt ooff ssaammpplliinngg iiss nneecceessssaarryy,, iitt iiss oofftteennuunncclleeaarr wwhhyy aa cclleeaarr aanndd pprree-ddeeffiinneedd ssaamm-pplliinngg pprroocceessss iiss ssoo iimmppoorrttaanntt,, aanndd iinnddeeeedd,,wwhhyy iitt iiss nneeeeddeedd aatt aallll.. AAss aa ccoonnsseeqquueennccee,,ssttaattiissttiicciiaannss aarree oofftteenn ccaalllleedd iinn,, ssoommeettiimmeess aatttthhee llaasstt mmiinnuuttee,, ttoo sseelleecctt tthhee mmoosstt ‘‘aapppprroopprrii-aattee’’ ssaammppllee uussiinngg ssoommee ssoorrtt ooff ccoommpplleexx ccoomm-ppuutteerr aallggoorriitthhmm bbeehhiinndd cclloosseedd ddoooorrss.. TThheemmeetthhoodd ooff sseelleeccttiinngg tthhee ssaammppllee rreemmaaiinnss aasseeccrreett ttoo tthhoossee oouuttssiiddee tthhee rreeaallmmss ooff ssttaattiissttiiccss,,tthheerreebbyy mmaakkiinngg tthhee pprroocceessss ooff ssaammpplliinngg aannddssttaattiissttiiccss iinn ggeenneerraall,, eevveerr mmoorree eelluussiivvee ttoo tthheennoonn-ssttaattiissttiicciiaann aanndd tthhuuss eennssuurriinngg tthhaatt tthheeffuuttuurree pprroossppeeccttss ooff ccaarreeeerrss ffoorr ssttaattiissttiicciiaannssrreemmaaiinn aass aattttrraaccttiivvee aass eevveerr.. IIss tthhee pprroocceessss ooffssaammpplliinngg ddiiffffiiccuulltt ttoo uunnddeerrssttaanndd aanndd iimmppllee-mmeenntt?? TThhee aannsswweerr iiss NNOO.. IInn ffaacctt,, tthhee ccoonncceeppttooff ssaammpplliinngg iiss ssiimmppllee aanndd tthhee mmeetthhooddss ffaarrffrroomm ccoommpplliiccaatteedd.. IItt iiss oofftteenn tthhee ccaassee,, hhooww-eevveerr,, tthhaatt ttooppiiccss ooff aa mmaatthheemmaattiiccaall nnaattuurreeaarree ppoooorrllyy eexxppllaaiinneedd aanndd aarree ccoonnsseeqquueennttllyyppoooorrllyy uunnddeerrssttoooodd.. IItt iiss tthhuuss ssoommeettiimmeess nneecc-eessssaarryy tthhaatt aann aapppprroopprriiaattee iinnttrroodduuccttiioonn aannddhheellppffuull eexxppllaannaattiioonn rreeggaarrddiinngg tthheeiirr uussee,,pprroocceessss ooff iimmpplleemmeennttaattiioonn,, aanndd ppoossssiibbllee bbeenn-eeffiittss bbee mmaaddee aavvaaiillaabbllee.. CCooppyyrriigghhtt ©© 22000011JJoohhnn WWiilleeyy && SSoonnss,, LLttdd..

Statistical sampling is the practice of selecting asample of items with the intention of being ableto infer with a certain amount of confidence oneor more characteristics regarding a larger popu-lation of which the sample is representative.

process of sampling takes time, it uses resources,it costs money, etc. Perhaps, the excuse most oftenused, especially regarding statistics in general, isthat ‘it’s only the statisticians that will under-stand it anyway!’ If we carefully look at the abovehighlighted paragraph, however, the question of“Why should we do it?” has already begun to beaddressed. Consider the components of the high-lighted definition of statistical sampling.

Firstly, ‘Statistical sampling is the practice ofselecting a sample of items …,’ implies that weneed to select a sample. That means a selection ofitems that is invariably smaller in number thanthe total number of items. The issue of time,resources, and money therefore, is already begin-ning to be addressed, as the amount of these pre-cious commodities is far less for a process ofselecting and reviewing a sample than for aprocess whereby all items would be observed.

Secondly, ‘… with the intention of being able toinfer with a certain amount of confidence ... , ’means that we want to infer something. The wholeconcept of sampling and statistics in general isredundant unless something of relative impor-tance (that is, relative to the cost of the experi-ment) is to be inferred from the data collected.There is little point in taking the trouble of select-ing a sample if the data are not to be analysed andsomething subsequently concluded. Anotherpoint is that a certain amount of confidence willsurround the inferential assessment. Confidencein statistics is measured using probability and isable to provide a level of ‘comfort’ or ‘trust’regarding whether the results or messagesobtained from the data collected should bebelieved or not. Often statisticians will refer tothis confidence in percentage terms, e.g. 95% or99% confidence. It is in the interest of the exper-imenter to ensure that the level of confidence surrounding the results is as high as humanlypossible, since it is decisions that will be taken inthe future that are reliant on the accuracy andcredibility of the results.

Would it be reasonable, however, to assumethat making a wrong decision following an exper-iment is only going to occur by chance? A decisionthat is taken following an experiment may not be

all that important and a level of chance of makingthe wrong decision does not have to be quantified,i.e. a level of comfort surrounding the decision isnot required. Other decisions, however, requirethat a certain degree of comfort in making theright decision is made available to the decision-maker. Consider the decision surrounding thediagnosis of a life-threatening disease such asAIDS. The procedures used to diagnose whetheran individual has contracted the HIV virus musttry to avoid providing conclusions in the form offalse positive and false negative diagnoses. Theprobability, or confidence, of making the correctdiagnosis must therefore be as high as humanlypossible if damaging mistakes are to be avoided.This example is a dramatic one, but it illustratesthe point that a certain degree of confidence sur-rounding experimental results and subsequentdecisions (or in this case a diagnosis) is oftenmandatory. The International Organisation forStandardisation (ISO) document ISO 2859-1(1989) [1], which details the sampling proceduresthat are to be followed for inspection, quantifiesconfidence by suggesting the use of an ‘acceptablequality level (AQL)’, which is intended to providean upper limit of ‘risk to the consumer’.

Thirdly, consider the last part of the defini-tion, that is ‘… one or more characteristicsregarding a larger population of which the sam-ple is representative.’ The point of making aninference regarding a larger population has beenbriefly discussed above. It is sometimes too costly,and indeed most of the time not possible, to takea measurement of every item in the population.Therefore to ensure that the experiment remainsfeasibly practical, a representative subset of thispopulation must be selected and with it the con-cept of sampling is born.

Sampling from aPopulation

To understand the concept of sampling, we mustfirst be able to understand the definition of a pop-ulation. A population can be thought of as a groupof people (or items) about which we would like to

158 M. Scott

Copyright © 2001 John Wiley & Sons, Ltd. Qual Assur J 2001; 55, 157–161.

obtain information [2]. According to ISO 2859-1,a population or ‘lot’ is ‘… a collection of units ofproduct from which a sample shall be drawn andinspected to determine conformance with theacceptability criteria …’. This can be anythingthat the sampler wishes to draw conclusionsabout, e.g. cars using a particular stretch of roadover 24 h, or the height of women in GreatBritain. In the field of quality assurance, particu-lar examples may include faulty componentswithin a manufacturing process, mistakes madeon a clinical report form, poor quality X-rays atEuropean clinical trial investigator sites, all ofwhich would be referred to in ISO 2859-1 as ‘non-conformities’.

Although the idea of a population is a rathereasy one to grasp, its exact definition must beconsidered carefully. The sampler must firstascertain what questions he/she wishes to answer,as without this basic step accomplished, the correct population with which the answers will beprovided will be impossible to establish. In somecases this is straightforward: e.g. two clinical drugmanufacturing plants A and B producing blis-tered packets of drug X. If we wish to investigatemanufacturing faults of the blistered packets ofdrug X (non-conformities) occurring in factory A,then our population will be all blistered packetsof drug X occurring in factory A. This would notbe the same population as all blistered packets of drug X, however, as faults may also occur infactory B.

The chosen population relates directly back tothe question that we wished to answer at thebeginning, thus making the population straight-forward to define. In contrast, the definition of apopulation in the field of pharmaceutical sciencefor example is far more complicated. Before con-ducting a clinical trial on humans, the question ofwhom the drug is likely to benefit is the first ques-tion to be answered in the process of defining thepopulation. Questions regarding disease history,medication history, gender, age, etc. to name afew, have to be addressed in order to decide uponthe population for which the drug is considered tohave maximum potential. Populations can there-fore be extremely specific.

The fundamental point to remember regardingthe definition of the population is that all conclu-sions from the ensuing experiment relate only tothe population that was defined initially and to noother. In the factory example, conclusions couldnot be made about the manufacturing process infactory B if the population was defined to be offactory A. Likewise, a pharmaceutical companywould not be able to make conclusions regardingthe efficacy of a drug on patients with stomachcancer if the population was defined to bepatients with lung cancer. All this sounds intu-itive and to some extent obvious, but getting thefirst step right in defining the population is fun-damental to progressing with a good samplingdesign and eventually providing accurate answersto the questions posed.

From Population to Sample

The sample is simply a group of people or itemsfrom the population of interest e.g. 50 cars usingthe stretch of road, 1000 bolts produced by a factory, 800 patients with stomach cancer, etc. Asit is invariably impossible or simply not costeffective (or even destructive, e.g. safety ex-periments of cars) to sample the whole popula-tion, the size of the sample is almost always a fraction of that of the population as illustrated inFigure 1. Indeed, taking a small sample can oftenlead to improved precision regarding the para-meter of interest as more care can be investedinto the measurement of each sample unit,

Copyright © 2001 John Wiley & Sons, Ltd. Qual Assur J 2001; 55, 157–161.

Statistical Sampling – Practical Advice for QA 159

Figure 1. From population to sample

thereby improving accuracy. Taking measure-ments from the whole population is likely to behugely time consuming, involving many people,thereby introducing a large source of variabilityand possible error which will ultimately bereflected in the precision of the parameter ofinterest upon completion of the analysis.

How Large Must theSample Be?

This is probably the most frequently asked ques-tion in the field of sampling as not only does ithave an impact on the precision of the parameterof interest upon which the decisions are based,but also on time and therefore the money to beinvested. Ultimately, the sample should be aslarge or small as the sampler wishes. It is notintended here to provide the reader with complexformulae in order to calculate the optimal samplesize, as many methods have evolved over theyears, all specifically designed for their statisticalanalysis counterparts. Many papers are dedicatedto the ideas of sample size and it would be wrongto attempt to provide a quick and easy solutionhere.

In principle, as larger samples provide moreinformation in terms of the number of data thana small sample, it is the larger samples that aregenerally more attractive (at least for those indi-viduals who are not responsible for the budget).ISO 2859-1 provides some good advice withregard to this topic. In particular, it mentions that‘The amount of information about the quality ofthe lot gained from examining samples drawnfrom the lot depends upon the absolute size of thesamples, not upon the percentage of the lot, pro-vided the lot is large relative to the sample ...’.This is to say, for example, that 50 items takenfrom a population of 10 000 will provide as muchinformation as 50 items taken from a populationof 5000. At first glance this may appear somewhatstrange as it is intuitive to presume that 50 from5000 provides twice as much information as 50from 10 000. The point, however, is that 50 itemswere selected. The size of the original population

does not play any role in the accuracy providedby the sample. It is the size of the sample that dic-tates its own accuracy.

If the population is extremely large, it is likelythat any future decisions will have more of animpact than if the population is small. In thiscase, it is probably advisable to take a larger sam-ple to provide more precision for the parameterof interest and therefore more comfort for thedecision-maker. As ISO 2859-1 states ‘… whenthere is more at stake, it is more important tomake the right decision.’

Sometimes, however, it is not possible to attaina large sample. This may be due to lack of timeand resources, or because the items of interest areno longer available to be sampled (e.g. laboratorysamples during an audit of a clinical trial investi-gator site), or because the process of sampling isactually destructive (e.g. accuracy of ammunitionfollowing firing). When this is the case, all effortsshould be made to remove as much variabilityfrom the sampling procedure as possible. Re-moval of variability will, by definition, improveprecision.

In designing a sampling scheme, it is criticalthat the sample be representative [3] of the pop-ulation for which answers are sought. The meth-ods for attaining a representative sample are notdiscussed here but it is sufficient to mention that,once the population is defined, the samplingscheme must be designed in such a way that sam-ples are actually taken from this population.Using the examples above: on a particular stretchof road, cars should be sampled across 24 h andnot, for example, only between 9am and 6pm;regarding the manufacturing example, items mustbe sampled from the manufacturing plant A andnot plant B. Again, all this seems to be ratherintuitive. However, when factors such as time andmoney dictate sampling, the temptation to samplefrom plant B due to plant A being closed for aweek may become significant.

The point is, as soon as we sample from a pop-ulation other than the one specified in advance,we are no longer justified in making conclusionson our population of interest as our sample hasembraced a larger population of which the one

160 M. Scott

Copyright © 2001 John Wiley & Sons, Ltd. Qual Assur J 2001; 55, 157–161.

that interests us is just a part. Figure 2 depicts thelogical thought processes that should ideally befollowed when designing a sampling scheme. Itserves to highlight how the person designing thesampling procedure should be thinking, ratherthan to provide clear instructions regarding howa sample should be collected.

Conclusion

It was the intention of this paper to provide anintroduction to statistical sampling. It discussedwhy sampling is sometimes necessary, and identi-fied some fundamental conditions that should beconsidered when designing the basics of a sam-pling procedure. These include, amongst others,correctly understanding the idea of populationand sample, and having a clear idea of the ques-tions that are to be answered from the informa-tion provided by the sample.

Finally, regarding the field of quality assur-ance, it has been shown how the basic concepts ofsampling relate to the accepted internationalstandards. It has also been suggested that, in afield where the requirement to follow sound sta-tistical principles is perhaps not as strict as inother fields (such as in phase III clinical trials),correct statistical sampling does nonetheless haveits rewards.

References

1. International Organization for Standardization. ISO 2859-1

Sampling Procedures for Inspection by Attributes. Part 1:

Sampling Plans Indexed by Acceptable Quality Level (AQL) For

Lot-by-Lot Inspection. 1989.

2. Harper WM. Statistics (3rd edn). Macdonald and Evans Ltd:

Plymouth, 1977.

3. Goodman R. Statistics. The English Universities Press Ltd:

London, 1966.

Copyright © 2001 John Wiley & Sons, Ltd. Qual Assur J 2001; 55, 157–161.

Statistical Sampling – Practical Advice for QA 161

Figure 2. First thought processing when designing a sampling scheme