the run test

7/29/2019 The Run Test

1/15

The Run Test

Let's start from the beginning... it seems reasonable to think that before wecan derive a Run Test, we better know what actually constitutes a run.

What is a run?

Let's suppose we have n1 observations of the random variableX,and n2 observations of the random variable Y. Suppose we combine the twosets of independent observations into one larger collectionofn1 + n2 observations, and then arrange the observations in increasing orderof magnitude. If we label from which set each of the ordered observationsoriginally came, we might observe something like this:

wherexdenotes an observation of the random variableXand ydenotes anobservation of the random variable Y. (We might observe this, for example, iftheXvalues were 0.1, 0.4, 0.5, 0.6, 0.8, and 0.9, and the Yvalues were 0.2,0.3, 0.7, 1.0, 1.1, and 1.2). That is, in this case, the smallest of all of theobservations is anXvalue, the second smallest of all of the observations isa Yvalue, the third smallest of all of the observations is a Yvalue, the fourth

smallest of all of the observations is anXvalue, and so on. Now, each groupof successive values ofXand Yis what we call a run. So, in this example,we have six runs. If we instead observed this ordered arrangement:

we would have three runs. And, if we instead observed this orderedarrangement:

we would have eight runs.


2/15

Why runs?

The next obvious question is in what way might knowing the number of runsbe helpful in testing the null hypothesis of the equality of the distributions F(x)and G(y). Let's investigate that question by taking a look at a few examples.

Let's start with the case in which the distributions are equal. In that case, wemight observe something like this:

In this particular example, there are eight runs. As you can see, this kind of apicture suggests that when the distributions are equal, the number of runs willlikely be large.

Now, let's take a look at one way in which the distribution functions could beunequal. One possibility is that one of the distribution functions is at least asgreat as the other distribution function at all points z. This situation might looksomething like this:


3/15

In this particular example, there are only two runs. This kind of a situation

suggests that when one of the distribution functions is at least as great as theother distribution function, the number of runs will likely be small. Note thatthis is what the distribution functions might look like if the median ofYwasgreater than the median ofX.

Here's another way in which the distribution functions could be unequal:

In this case, the medians ofXand Yare nearly equal, but the variance ofYismuch greater than the variance ofX. In this particular example, there are onlythree runs. Again, in general, when we have this type of situation, we wouldexpect the number of runs to be small.


4/15

The above three examples give a pretty clear indication that we're ontosomething with the idea of using runs to test the null hypothesis of the equalityof two distribution functions. If the number of runs is smaller than expected, itseems we should reject the null hypothesis, because a small number of runssuggests that there are differences in either the location or the spread of the

two distributions. Now, as is usually the case when conducting a hypothesistest, in order to quantify "smaller than expected," we're going to need to knowsomething about the distribution ofR, the number of runs.

What is the p.m.f. ofR?

Let's let the random variable Rdenote the number of runs in the combinedordered sample containing n1 observations ofXand n2 observations ofY.With possible values, such as 2, 3, and so on, Ris clearly discrete. Now, if thenull hypothesis is true, that is, the distribution functions are equal, all of the

possible permutations of theX's and the Y's in the combined sample areequally likely. Therefore, we can use the classical approach to assigning theprobability that Requals a particular value r. That is, to find the distributionofR, we can find:

for all of the possible values in the support ofR. (Note that the supportdepends on the number of observations in the combined sample. We doknow, however, that Rmust be at least 2.) As is usually the case, thedenominator is the easy part. If we have n1 + n2 positions:

in which we can choose to place the n1 observedxvalues, then the totalnumber of ways of arranging thex's and y's is:

(n1+n2n1)


5/15

Note that once thexvalues are placed in their positions, theremaining n2 positions must contain the yvalues. We just as easily couldplace the yvalues first, and then the remaining n1 positions must contain the xvalues.

Now for the numerator. That's the messy part! Before we even begin toattempt to generalize a formula for the quantity in the numerator, it might bebest to take a look at a concrete example.

Example

Suppose we have n1 = 3 observations of the random variableXand n2 = 3observations of the random variable Y. What is the p.m.f. ofR, the number ofruns?

Solution. Put your seatbelt on. If not your seatbelt, then at least your

bookkeeping hat. This is going to get a bit messy at times. So... let's start withthe easy part first... the denominator! If we combine the two samples, we havea total of 3+3 = 6 observations to arrange. Therefore, there are:

(3+33)=(63)=6!3!3!=20

possible ways of arranging thex's and y's. Now, that's a small enough numberthat we can actually enumerate all 20 of the possible arrangements. Here theyare:


6/15

Now, as the above table suggests, of the 20 possible arrangements, there aretwo ways of getting R= 2 runs. Therefore:

P(R=2)=220

And, of the 20 possible arrangements, there are four ways of getting R= 3runs. Therefore:

P(R=3)=420

Piece of cake! What was all this talk about messy bookkeeping? With thesnap of a finger, we can determine the entire p.m.f. ofR:

Note that the probabilities add to 1. That's a good sign! But finding the specificp.m.f. ofRwas not the point of this example. We were going to use thisexample to try to learn something about finding the p.m.f. ofRin general. The


7/15

only reason why we were able to find this p.m.f. ofRwith such ease wasbecause we could enumerate all 20 of the possible outcomes. What if we can'tdo that? That is, what if there were so many possible arrangements that itwould be crazy to try to enumerate them all? Well, we would go way back towhat we did in Stat 414... we would derive a general counting formula! That's

what we're working towards here.

Let's take a look at a case in which ris even, r = 4, say. If you think about it,the only way we can get four runs is if the n1 = 3 values ofXare broken upinto 2 runs, which can be done in either of two ways:

and the n2 = 3 values ofYare broken up into 2 runs, which can be done ineither of two ways:

Now, it's just a matter of putting the 2 runs ofXand the 2 runs ofYtogether tomake a total of 4 runs in the sequence. Well, there are two ways of getting 2runs ofXand two ways of getting 2 runs ofY. Therefore, the multiplicationrule tells us that we should expect there to be 22 = 4 ways of getting four

runs when the samples are combined. There's just one problem with thatcalculation, namely that there are 2 ways of starting the sequence off. That is,we could start with anXrun:


8/15

or we could start with a Yrun:

So, the multiplication rule actually tells us that there are 222 = 8 ways ofgetting four runs when we have n1 = 3 values ofXand n2 = 3 values ofY.

And, we've just enumerated all eight of them!

Now, let's try to generalize the above process. We started by considering acase in which rwas even, specifically, r= 4. Note that r is even impliesthat r= 2k, for a positive integerk. (Ifr= 4, for example, then k= 2.) Now, theonly way we can get r= 4 runs is if the n1 = 3 values ofXare broken upinto k= 2 runs and the n2 = 3 values ofYare broken up into k= 2 runs. Wecan form the k= 2 runs of the n1 = 3 values ofXby inserting k1 = 1 dividerinto the n1 1 = 2 spaces between theXvalues:


9/15

When the divider is here:

we create these two runs:

And, when the divider is here:

we create these two runs:

Note that, in general, there are:(n11k1)

ways of inserting k1 dividers into then1 1 spaces between theXvalues,with no more than one divider per space.


10/15

Now, we can go through the exact same process for the Yvalues. It shouldn'tbe too hard to see that, in general, there are:

(n21k1)

ways of inserting k1 dividers into then2 1 spaces between theYvalues, with nomore than one divider per space.

Now, if you go back and look, once we broke up theXvalues into k= 2 runs, and

the Yvalues into k= 2 runs, the next thing we did was to put the two sets of two runs

ofXand two sets of two runs ofYtogether. The multiplication rule tells us that there

are:(n11k1)(n21k1)

ways of putting the two sets of runs together to form r= 2kruns beginning with a run

ofx's. And, the multiplication rule tells us that there are:(n21k1)(n11k1)

ways of putting the two sets of runs together to form r= 2kruns beginning with a runofy's. Therefore, the total number of ways of getting r= 2kruns is:(n11k1)(n21k1)+(n21k1)(n11k1)=2(n11k1)(n21k1)

And, therefore putting the numerator and the denominator together, we get:P(R=2k)=2(n11k1)(n21k1)(n1+n2n1)

when kis a positive integer.

Whew! Now, we've found the probability of getting rruns when ris even. What

happens ifris odd? That is, what happens ifr= 2k+1 for a positive integerk? Well,

let's consider the case in which r = 3, and therefore k= 1. If you think about it, there

are two ways we can get three runs... either we need k+1 = 2 runs ofx's and k= 1 runofy's, such as:

and

or we need k= 1 run ofx's and k+1 = 2 runs ofy's:


11/15

and

Perhaps you can see where this is going? In general, we can form k+1 runs of

the n1 values ofXby inserting kdividers into the n11 spaces between theXvalues,with no more than one divider per space, in:

(n11k)

ways. Similarly, we can form kruns of the n2 values ofYin:(n21k1)

ways. The two sets of runs can be placed together to form 2k+1 runs in:(n11k)(n21k1)

ways. Likewise, k+1 runs of the n2 values ofY, and kruns of the n1 values ofXcan be

placed together to form:(n21k)(n11k1)

sets of 2k+1 runs. Therefore, the total number of ways of getting r= 2k+1 runs is:(n11k)(n21k1)+(n21k)(n11k1)

And, therefore putting the numerator and the denominator together, we get:P(R=2k+1)=(n11k)(n21k1)+(n21k)(n11k1)(n1+n2n1)

Yikes! Let's put this example to rest!

Let's summarize what we've learned.

Summary. The probability thatR, the number of runs, takes on a particular

value ris:

P(R=2k)=2(n11k1)(n21k1)(n1+n2n1)

ifris even, that is, ifr= 2kfor a positive integerk. The probability thatR, the

number of runs, takes on a particular value ris:

P(R=2k+1)=(n11k)(n21k1)+(n21k)(n11k1)(n1+n2n1)

ifris odd, that is, ifr= 2k+1 for a positive integerk.

Now, let's wrap up our development of the Run Test with an example.


12/15

Example

LetXand Ydenote the times in hours per weeks that students in two differentschools watch television. Let F(x) and G(y) denote the respective distributions.To test the null hypothesis:

H0:F(z)=G(z)

a random sample of eight students was selected from each school, yieldingthe following results:

What conclusion should we make about the equality of the two distribution

functions?

Solution. The combined ordered sample, with thexvalues in blue andthe yvalues in red, looks like this:

Counting, we see that there are 9 runs. We should reject the null hypothesis ifthe number of runs is smaller than expected. Therefore, the critical regionshould be of the form rc. In order to determine what we should set the valueofcto be, we'll need to know something about the p.m.f. ofR. We can use theformulas that we derived above to determine various probabilities. Well,with n1 = 8, n2 = 8, and k= 1, we have:


13/15

P(R=2)=2(70)(70)(168)=212,870=0.00016

and:

P(R=3)=(71)(70)+(71)(70)(168)=1412,870=0.00109

(Note that Table I in the back of the textbook can be helpful in evaluating thevalue of the "binomial coefficients.") Now, with n1 = 8, n2 = 8, and k= 2, wehave:

P(R=4)=2(71)(71)(168)=9812,870=0.00761

and:

P(R=5)=(72)(71)+(72)(71)(168)=29412,870=0.02284

And, with n1 = 8, n2 = 8, and k= 3, we have:

P(R=6)=2(72)(72)(168)=88212,870=0.06853

Let's stop to see what we have going for us so far. Well, so far we've learnedthat:

P(R6)=0.00016+0.00109+0.00761+0.02284+0.06853=0.1002

Hmmm. That tells us that if we set c= 6, we'd have a 0.1002 probability ofcommitting a Type I error. That seems reasonable! That is, let's decide toreject the null hypothesis of the equality of the two distribution functions if the

number of observed runs r 6. It's not... we observed 9 runs. Therefore, wefail to reject the null hypothesis at the 0.10 level. There is insufficient evidenceat the 0.10 level to conclude that the distribution functions are not equal.

A Large-Sample Test

As our work in the previous example illustrates, conducting a single run testcan be quite extensive in the calculation front. Is there an easier way?Fortunately, yes... that is, providing n1 and n2are large. Typically, we considerthe samples to be large ifn1 is at least 10 and n2 is at least 10. If the samples

are large, then the distribution ofRcan be approximated with a normallydistributed random variable. That is, it can be shown that:

Z=RRVar(R)

follows an approximate standard normal N(0, 1) distribution with mean:

R=E(R)=2n1n2n1+n2+1


14/15

and variance:

Var(R)=2n1n2(2n1n2n1n2)(n1+n2)2(n1+n21)

.

Because a small number of runs is evidence that the distribution functions areunequal, the critical region for testing the null hypothesis:

H0:F(z)=G(z)

is of the form z z, where is the desired significance level. Let's take alook at an example.

Example

A charter bus line has 48-passenger buses and 38-passenger buses.WithXand Ydenoting the number of miles traveled per day for the 48-passenger and 38-passenger buses, respectively, the bus company isinterested in testing the equality of the two distributions:

H0:F(z)=G(z)

The company observed the following data on a random sample ofn1 = 10buses holding 48 passengers and n2 = 11 buses holding 38 passengers:

Using the normal approximation to R, conduct the hypothesis test at the 0.05level.


15/15

Solution. The combined ordered sample, with thexvalues in blue andthe yvalues in red, looks like this:

Counting, we see that there are 9 runs. Using the normal approximation to R,with n1 = 10 and n2 = 11, we have:

R=2n1n2n1+n2+1=2(10)(11)10+11+1=11.476

and:

Var(R)=2n1n2(2n1n2n1n2)(n1+n2)2(n1+n21)=2(10)(11)[2(10)(11)1011](10+11) 2(10+111)=4.9637

Now, we observed r= 9 runs. Therefore, the approximate P-value, using ahalf-unit correction for continuity, is:

PP[Z9.511.4764.9637 ]=P[Z0.89]=0.187

We fail to reject the null hypothesis at the 0.05 level, because the P-value isgreater than 0.05. There is insufficient evidence at the 0.05 level to concludethat the two distribution functions are unequal.

the run test

Documents