Computers & Operations Research 40 (2013) 2569–2584

Contents lists available at SciVerse ScienceDirect

Computers & Operations Research

0305-05

http://d

n Corr

E-m

journal homepage: www.elsevier.com/locate/caor

Allocating job-shop manpower to minimize Lmax: Optimality criteria,search heuristics, and probabilistic quality metrics

Benjamin J. Lobo a, Thom J. Hodgson a, Russell E. King a, Kristin A. Thoney b,James R. Wilson a,n

a Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Campus Box 7906, Raleigh, NC 27695-7906, USAb Department of Textile and Apparel Technology and Management, North Carolina State University, Campus Box 8301, Raleigh, NC 27695-8301, USA

a r t i c l e i n f o

Available online 27 February 2013

Keywords:

Job shop scheduling

Dual resource constrained systems

Maximum lateness

Worker allocation

48/$ - see front matter & 2013 Elsevier Ltd. A

x.doi.org/10.1016/j.cor.2013.02.008

esponding author. Tel.: þ1 919 606 4031.

ail address: jwilson@ncsu.edu (J.R. Wilson).

a b s t r a c t

We address questions raised by Lobo et al. in 2012 regarding the NP-hard problem of finding an optimal

allocation of workers to machine groups in a job shop so as to minimize Lmax, the maximum job

lateness. Lobo et al. formulated a lower bound on Lmax given a worker allocation, and an algorithm to

find an allocation yielding the smallest such lower bound. In this article we establish optimality criteria

to verify that a given allocation corresponds to a schedule that yields the minimum value of Lmax.

For situations in which the optimality criteria are not satisfied, we present the Heuristic Search

Procedure (HSP), which sequentially invokes three distinct search heuristics, the Local Neighborhood

Search Strategy (LNSS), Queuing Time Search Strategy 1 (QSS1), and Queuing Time Search Strategy 2

(QSS2), before delivering the best allocation encountered by LNSS, QSS1, and QSS2. HSP is designed to

find allocations allowing a heuristic scheduler to generate schedules with a smaller value of Lmax than

that achieved via the allocation yielding the final lower bound of Lobo et al. Comprehensive

experimentation indicated that HSP delivered significant reductions in Lmax. We also estimate a

probability distribution for evaluating the quality (closeness to optimality) of an allocation delivered by

a heuristic search procedure such as HSP. This distribution permits assessing the user’s confidence that

a given allocation will enable the heuristic scheduler to generate its best possible schedule—i.e., the

schedule with the heuristic scheduler’s smallest achievable Lmax value.

& 2013 Elsevier Ltd. All rights reserved.

1. Introduction

Job shops in which both machines and workers form con-straints are referred to in the literature as dual resourceconstrained (DRC) systems [1,2]. DRC systems more closely modelreal life by accounting for the constraints imposed on systemoperation by the limited availability of workers. In a job shop withN jobs and M machines on which to process the jobs, the JM99Lmax

single resource constrained job shop scheduling problem is to finda schedule that minimizes Lmax, the maximum job lateness. Thisproblem is known to be NP-hard [3]. Adding the worker con-straint makes the problem even harder to solve, because itrequires simultaneous optimization of the schedule and theallocation of workers to machine groups in the job shop.

Beginning with Nelson [4], DRC job shops have receivedconsiderable attention from the scheduling research community;see the reviews by Treleven [1], Gargeya and Deane [5], andHottenstein and Bowman [6]. The previous research has focused

ll rights reserved.

on gauging the impact of the second constrained resource on jobshop performance, and can be categorized for the most part bythe worker assumptions. Examples of worker assumptionsinclude worker allocation rules (where and when a worker canbe transferred); worker skill set (e.g., the types of machines that aworker is capable of operating); worker efficiency; and in themore recent literature, worker learning, fatigue, and forgetfulness(i.e., skills and efficiencies that deteriorate if not used) [7–10].However, little research could be found that determines anoptimal way to allocate a fixed number of workers in a DRCjob shop.

For the DRC job shop scheduling problem, Lobo et al. [11]recently formulated a lower bound on Lmax that accounts for bothconstrained resources. Given an allocation W of workers tomachine groups, the authors derived a lower bound LBW on Lmax

for that worker allocation. They also developed an algorithm tofind a worker allocation Wn that yields the smallest value of LBWover all feasible values of W. The authors’ final lower bound onLmax, LBWn , provides a benchmark against which heuristic solu-tions to this NP-Hard problem can be compared. However, theultimate objective is to find an optimal allocation, defined to bean allocation associated with a schedule that minimizes Lmax for

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–25842570

the job shop. Lobo et al. [11] observed randomly generatedproblem instances in which the schedule generated by the VirtualFactory (a heuristic scheduler developed by Hodgson et al. [12])using an allocation other than Wn had a smaller Lmax value thanthat for the schedule based on allocation Wn, showing that ingeneral Wn is not guaranteed to be an optimal allocation ofworkers to machine groups.

In this article we present research that addresses the objectiveof finding an optimal or nearly optimal worker allocation forthe DRC job shop scheduling problem through the followingcontributions:

1.

Developing and justifying criteria which are easy to check inpractice and which are sufficient to guarantee that the workerallocation Wn is optimal—i.e., Wn not only yields the smallestfeasible value of LBW but also corresponds to a schedule thatminimizes Lmax;

2.

Formulating a heuristic search procedure to find optimal (or atleast better) allocations in practical applications for which theoptimality criteria of item 1 are not satisfied; and

3.

Providing a method for probabilistically evaluating the quality(closeness to optimality) of a given allocation, and also assessingthe user’s degree of confidence that the Virtual Factory can use thegiven allocation to generate its best possible schedule.

If the optimality criteria of item 1 are not satisfied for a givenproblem instance, then we seek a ‘‘Virtual Factory–best’’ alloca-tion using the search heuristic of item 2. Such an allocationenables the Virtual Factory to generate a schedule with theminimal value of Lmax that can be achieved by the Virtual Factoryover all feasible allocations of workers to machine groups.In certain circumstances, the method of item 3 can be used tomake the following statement: for a user-specified significancelevel oAð0,1Þ, the user’s degree of confidence is approximately100ð1�oÞ% that a given worker allocation will enable the VirtualFactory to generate a VF-best schedule for the DRC job shopscheduling problem at hand. It must be pointed out that thesolution approach developed here would work with any otherheuristic scheduler chosen, although the results obtained woulddiffer based on the quality of the heuristic scheduler chosen.

Our approach to allocation of fully cross-trained workers wasmotivated by our examination of, and involvement with, theoperations in a major high-end furniture manufacturer, a large-scale apparel producer, and the US Navy’s Aviation Depots. Ineach case, cross-trained workers were allocated on an ad hocbasis; and in each of these disparate organizations, there was aclearly recognized need for a more systematic and effectiveapproach to the allocation of workers.

The rest of this article is organized as follows. Section 2provides the background information required for precise speci-fication of the DRC job shop scheduling problem. In Section 3 weformulate and justify criteria for checking whether a given workerallocation is optimal. In the case that neither optimality criterionis satisfied, in Section 4 we develop a number of heuristicstrategies to search for an optimal allocation, and Section 5provides a summary of the results of our experimental perfor-mance evaluation of the heuristics. In order to probabilisticallyevaluate the quality of the allocation delivered by the heuristics,in Section 6 we present a method for estimating certain prob-ability distributions from which we can assess the user’s degree ofconfidence that a given worker allocation will enable generationof a VF-best schedule. Section 7 recapitulates the main findingsof this work and provides recommendations for future work. TheOnline Supplement to this article contains the following: (a) theproof of one of the optimality criteria found in Section 3; (b) a

formulation of the confidence-interval estimation method used toevaluate the performance of the proposed heuristic search pro-cedure provided in Section 5; and (c) the full set of resultssupporting the experimental performance evaluation summar-ized in Sections 5 and 6.

2. Background on the DRC job shop scheduling problem

In this article we make the following assumptions about theworkers in a DRC job shop:

�

There are fewer workers than machines in the job shop. � A machine group must have at least one worker assigned to it;

otherwise, the machine group cannot process any jobs.

� A machine requires one worker to operate it, and a worker

cannot operate more than one machine at a time; moreover,the worker must be present at a machine whilst processing ofa job is underway.

� The allocation of workers to machine groups, once determined,

is fixed.

� Every worker can be assigned to any machine group in the

job shop.

Machine groups arise naturally in a job shop; the machines thatperform similar operations will typically be located in physicalproximity of each other and will require the same basic skills andexpertise to operate.

The problem addressed in this article is denoted as theJM9W9Lmax DRC job shop scheduling problem [11], and is definedas follows. The DRC job shop has a total of N jobs and M machineson which to process those jobs. Every job j has a routing throughthe job shop, a processing time pjh at each machine h that the jobvisits on its route, and a due-date dj. Machines in the DRC jobshop that perform similar tasks are grouped together, where mgi

denotes the ith machine group consisting of the set of machinesMi for i¼ 1, . . . ,S, with S denoting the number of machine groupsin the job shop. Of the W workers in the job shop, wi workersare assigned to mgi so that 1rwir9Mi9 (where 9Mi9 denotesthe size of the set Mi) for i¼ 1, . . . ,S; and we have

PSi ¼ 1 wi ¼

WoM¼PS

i ¼ 1 9Mi9. The objective is to determine an allocationof workers to machine groups in the job shop that corresponds toa schedule yielding the minimal value of Lmax. Because theJM9W9Lmax DRC job shop scheduling problem is NP-Hard, aheuristic scheduler is used to generate the schedule given aparticular allocation W of workers to machine groups.

The Virtual Factory was developed by Hodgson et al. [12] aspart of their solution to the JM99Lmax single resource constrainedjob shop scheduling problem. Incorporating a transient, determi-nistic simulation of a job shop, the Virtual Factory is based on anapproach proposed by Vepsalainen and Morton [13]. The VirtualFactory is an iterative procedure that sequences jobs using a slackcalculation on the first iteration. On each subsequent iteration,the Virtual Factory sequences the jobs using an iteratively revisedslack calculation that includes queuing time estimates from theprevious iteration. Iterations continue until a previously com-puted lower bound on Lmax is achieved or until the number ofiterations reaches a prespecified limit; then the Virtual Factorydelivers the best schedule encountered over all iterations. Wechose the Virtual Factory as the heuristic scheduler because of itsproven track record in successfully generating optimal ornear-optimal schedules to job shop scheduling problems forwhich the primary objective is to minimize Lmax [14–19], butthe approach presented in this article would work with otherheuristic schedulers.

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–2584 2571

Lobo et al. [11] determine a lower bound on Lmax for a givenallocation W of workers to machine groups as follows. First, anetwork flow approach is used to find a lower bound on Lmax foreach individual machine group; then the lower bound LBW on Lmax

for the entire DRC job shop is taken to be the maximum of therespective lower bounds on Lmax for the individual machinegroups. Next the authors’ Lower Bound Search Algorithm (LBSA)is used to find the smallest such lower bound on Lmax for the jobshop taken over all feasible allocations of workers to machinegroups, i.e., LBWn . LBSA delivers the worker allocation Wn togetherwith the index k of a constraining machine group. Given aparticular allocation W of workers to machine groups, a constrain-ing machine group is a machine group whose individual lowerbound on Lmax is equal to the lower bound LBW on Lmax for theentire job shop.

Remark 1. For each constraining machine group associated witha given allocation W, that machine group’s lower bound on Lmax

coincides with LBW, the lower bound on Lmax for the entire DRC jobshop based on allocation W as detailed in the preceding paragraph.Therefore a constraining machine group is a good candidate forhaving an additional worker reassigned to it from some othernonconstraining machine group, so that the resulting new alloca-tion W0 might be expected to yield LBW0oLBW. If the latter inequal-ity holds, then we have some reason to expect that W0 maycorrespond to a schedule with a smaller value of Lmax thancould be obtained using allocation W. This is the main reasonfor focusing attention on the constraining machine groups asso-ciated with each allocation W that is considered in the followingdevelopment.

LBSA terminates at allocation Wn if either Condition 1 orCondition 2 is satisfied, where:

�

Condition 1 specifies that wkðWnÞ ¼ 9Mk9; i.e., under allocation

Wn the number of workers wkðWnÞ assigned to the currently

selected constraining machine group k is equal to the numberof machines in that machine group; and

� Condition 2 specifies that wkðW

nÞo9Mk9 and that for each

machine group jak, it is either infeasible, suboptimal, or atbest nonimproving to reassign a worker to the currentlyselected constraining machine group k from machine groupj; i.e., if we change the current allocation by reassigning tomachine group k one worker from any other machine group,then the lower bound on Lmax for the entire job shop will notdecrease (it will usually increase), and the machine grouplosing a worker will become (or remain) constraining.

Section 3 contains a more detailed explanation of the terminationconditions for LBSA. Lobo et al. [11] prove that LBSA is guaranteedto find an allocation yielding the smallest LBW value, whereseveral such allocations may exist; and in some (rare) cases,there may be several constraining machine groups associatedwith the allocation delivered by LBSA.

3. Optimality criteria

For each feasible allocation W, let LBW denote the associatedlower bound on Lmax for the job shop as defined in Lobo et al. [11]and in Section 2 of this article. Similarly, we let VFW denote thevalue of Lmax resulting from the schedule generated by the VirtualFactory using allocation W. A VF-best allocation (denoted by WVFB)is defined to be one for which a schedule generated by the VirtualFactory will have the minimum value of Lmax that can be achievedusing the Virtual Factory with all feasible allocations of workers

to machine groups

WVFB� arg min VFW

W is a feasible allocation; i:e:; 1rwiðWÞr9Mi9

for i¼ 1, . . . ,S, andXS

i ¼ 1

wiðWÞ ¼W

��������8>><>>:

9>>=>>;,

ð1Þ

where wiðWÞ denotes the number of workers allocated to machinegroup i under allocation W for i¼ 1, . . . ,S. We let VFWVFB denote thecorresponding VF-best value of Lmax.

Remark 2. If the right-hand side of Eq. (1) defines a set with twoor more elements, then allocation WVFB is understood to be anarbitrary member of that set. Note that a VF-best allocation is notnecessarily an optimal allocation, and that an optimal allocation isnot necessarily a VF-best allocation.

Theorem 1. If the allocation W satisfies

VFW ¼ LBWn , ð2Þ

then W is an optimal allocation—that is, associated with allocation Wthere is a schedule that yields the minimum possible value of Lmax for

the job shop; moreover, W is a VF-best allocation.

Proof. Eq. (2) immediately implies that W must be an optimalallocation. If W0 is any feasible allocation, then Eq. (2) andProposition 2 of Lobo et al. [11] ensure that VFW ¼ LBWn

rLBW0rVFW0 , showing that W is also a VF-best allocation. &

Next we formulate a sufficient condition for the allocation Wn

delivered by LBSA to be both an optimal and a VF-best allocation.For i¼ 1, . . . ,S, let LBmgi

ðwiðWnÞÞ denote the lower bound on Lmax

for machine group i when wiðWnÞ workers are assigned to that

machine group, and let

KðWnÞ � fi : 1r irS and LBmgi

ðwiðWnÞÞ ¼ LBWn g, ð3Þ

denote the index-set of constraining machine groups for alloca-tion Wn. If LBSA terminates with allocation Wn and with thecurrently selected constraining machine group kAKðWn

Þ, thenwe define the neighborhood of Wn with respect to the constrainingmachine group k as follows:

N ðWn,kÞ � f

f is a neighbor of Wn with wkðfÞ ¼wkðWnÞþ1 and

wjðfÞ ¼wjðWnÞ�1 for some jak; and wlðfÞ

¼wlðWnÞ for la j,k

�������8><>:

9>=>;:ð4Þ

Moreover, we define the associated neighborhood-optimal alloca-tion

fnðWn,kÞ � arg minfLBf : fAN ðWn,kÞg, ð5Þ

with the corresponding lower bound on Lmax

LBfnðWn ,kÞ �minfLBf : fAN ðWn,kÞg:

Remark 3. Throughout this article, W is used to denote anarbitrary allocation currently under discussion, while the symbolWn is reserved to denote the allocation delivered by LBSA whoseLmax value LBWn is the minimum value of LBW taken over all feasibleallocations W. Similarly, other lower case Greek letters are usedstrategically to distinguish between different types of allocations.For example, f is used throughout this article to denote anarbitrary allocation in the neighborhood N ðWn,kÞ of Wn definedby constraining machine group k, while fn

ðWn,kÞ is used todenote an allocation in that neighborhood whose Lmax valueLBfn

ðWn ,kÞ is the minimum value of LBf taken over all feasibleallocations f in that neighborhood. In general, a superscript isappended to a lower case Greek letter to denote an allocationwith some relevant property; for example, WHSP is used to denote

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–25842572

an allocation delivered by the Heuristic Search Procedure (HSP)detailed in Section 4.

Remark 4. On the right-hand side of Eq. (5), we take LBf �1 ifallocation f is infeasible, where the feasibility condition is givenon the right-hand side of Eq. (1). Moreover if the right-handside of Eq. (5) specifies a set consisting of more than oneneighborhood-optimal allocation, then fn

ðWn,kÞ is understood tobe an arbitrary member of that set.

Theorem 2. If LBSA terminates with an allocation Wn by satisfying

Condition 2 with a single constraining machine group k (i.e.,KðWnÞ ¼ fkgÞ such that

LBWn oLBfnðWn ,kÞ and VFWn rLBfn

ðWn ,kÞ, ð6Þ

then Wn is an optimal allocation; moreover, Wn is a VF-best allocation.

The proof of Theorem 2 appears in Section A1 of the OnlineSupplement.

Remark 5. In Theorem 2 the assumption of a single constrainingmachine group for allocation Wn might appear to be overly restrictive,but in our experience this is not the case. Of the 16,000 randomlygenerated instances of a symmetric job shop used in our experimen-tation, there were 11,712 cases in which LBSA delivered allocation Wn

by terminating because Condition 2 was satisfied; and of these lattercases, there were only 128 cases (i.e., 1.09% of the relevant probleminstances) in which LBSA terminated with more than one constrain-ing machine group. Of the 16,000 randomly generated instances of anasymmetric job shop used in our experimentation, there were 4974cases in which LBSA delivered allocation Wn by terminating becauseCondition 2 was satisfied; and of these latter cases, there were only34 cases (i.e., 0.68% of the relevant problem instances) in which LBSAterminated with more than one constraining machine group. Thus,we believe that Theorem 2 can be useful in practice for rapidverification that Wn is optimal.

Table 1 shows, for both an asymmetric and symmetric jobshop, and for staffing levels of 60%, 70%, 80%, and 90%, thepercentage of randomly generated problem instances in whichallocation Wn satisfied the hypotheses of either Theorem 1 orTheorem 2 (out of 200 randomly generated problem instances foreach combination of type of job shop, percent staffing, and due-date range). An in-depth description of the experimental job shopcan be found in Lobo et al. [11] and in Section 5.1 of this article.The table indicates that symmetric job shop problems with 70%,80%, and 90% staffing are especially difficult types of problems tosolve, as are asymmetric job shop problems with 70% staffing.

We observed that there are problem instances with thefollowing anomalous properties: (a) allocation Wn does not satisfyeither of the optimality criteria (i.e., the hypotheses of either

Table 1

Percentage of randomly generated problem instances for which allocation Wn

satisfies the optimality criteria of Theorem 1 or Theorem 2.

Due-date range Asymmetric job shop Symmetric job shop

Staffing level Staffing level

60% 70% 80% 90% 60% 70% 80% 90%

200 67.0 56.0 99.5 99.5 67.5 48.0 18.5 66.0

600 69.5 45.5 97.5 97.5 72.5 47.5 17.0 56.5

1000 67.5 40.5 92.5 92.0 75.0 47.5 13.0 46.0

1400 65.0 35.0 87.5 87.0 74.5 41.5 9.0 42.5

1800 60.5 35.5 86.5 86.5 75.0 38.5 6.5 36.0

2200 57.0 32.5 81.0 82.0 76.5 32.5 6.0 23.0

2600 55.5 28.5 75.5 76.0 73.0 27.0 2.5 14.5

3000 50.5 27.0 70.0 70.0 67.5 20.0 0.5 4.5

Theorem 1 or Theorem 2); and (b) there is no feasible allocation Wthat satisfies the hypothesis of Theorem 1. This observationmotivated the heuristic search strategies presented in the follow-ing section, which search for a VF-best allocation when allocationWn delivered by LBSA does not satisfy either optimality criteriapresented in this section.

4. Heuristic search strategies

If allocation Wn does not satisfy either of the optimality criteria,then other promising allocations must be identified and evaluated. Ifwe are lucky, then we will encounter a feasible allocation W0aWn

such that VFW0 ¼ LBWn , so Theorem 1 is satisfied. More often, however,we must hope that our search returns a VF-best allocation. In Section4.1 we present a partial enumeration strategy for finding a VF-bestallocation, and we summarize some computational results illustratingthe impracticality of this approach and underscoring the need foreffective heuristics to search for a VF-best allocation. Section 4.2details a heuristic that searches the local neighborhood of allocationWn, and Section 4.3 presents two heuristics that rely on queuing timeestimates to guide the search for a VF-best allocation. Section 4.4describes the combined heuristic search strategy HSP that incorpo-rates the three heuristics of Sections 4.2 and 4.3 and that is used in allexperimentation reported in Section 5.

4.1. A partial enumeration strategy for finding a VF-best allocation

For a given problem, to guarantee that an allocation is aVF-best allocation, we must use the Virtual Factory to evaluate eachfeasible allocation W for which LBWoVFWn . The partial enumerationstrategy outlined below attempts to minimize the number ofallocations that must be evaluated to find a VF-best allocation(which is sometimes also an optimal allocation). This strategyprogressively reduces the set of feasible worker allocations thatremain to be evaluated by iteratively reducing the upper bound VFWon acceptable values of LBg whenever a ‘‘VF-better’’ allocation W isencountered—i.e., whenever a newly encountered allocation Wyields the smallest value VFW encountered so far.

For each machine group i (1r irS) in the job shop, we firstcompute and store LBmgi

ðwiÞ for 1rwir9Mi9. This informationcan be used to determine the full set of feasible allocations thatyield the smallest LBW value for the DRC job shop,

C� arg minfLBW : all feasible Wg ¼ fW : LBW ¼ LBWn g:

The Virtual Factory can then be used to find an allocation

cn� arg minfVFc : cACg, so that VFcn ¼minfVFc : cACg:

If VFcn ¼ LBWn , then Theorem 1 ensures that cn is both anoptimal and a VF-best allocation. Otherwise, we define G as thefollowing set of allocations:

G� fg : LBWn oLBgoVFcn g:

Each allocation gAG must be evaluated using the Virtual Factory tocheck for the condition VFgoVFcn , showing that g is a betterallocation than cn. We can establish an iteratively reduced upperbound on LBg that requires only a subset of G to be evaluatedas follows: each time an allocation WAG is encountered suchthat VFW is strictly less than the smallest VF value found sofar, from that point on we evaluate only allocations gAG for whichLBWn oLBgoVFW. A VF-best allocation is then given by Eq. (1) so that

VFWVFB ¼minfVFW : all feasible Wg ¼minfVFW : WAG [ fcngg:

In general, the progressively reduced upper bound used in our partialenumeration strategy ensures that we stop substantially short of acomplete enumeration of all feasible worker allocations.

Table 2

Sample mean T and standard deviation ST of the execution time T taken (in

seconds) for the partial enumeration strategy in the symmetric job shop.

Due-date

range

60% Staffing 70% Staffing 80% Staffing 90% Staffing

T ST T ST T ST T ST

200 273.6 1029.4 288.2 639.7 284.4 416.0 166.6 196.0

600 95.5 122.3 191.1 328.9 331.3 437.6 308.7 1115.1

1000 83.8 75.8 195.0 309.6 449.7 581.4 437.4 1547.6

1400 96.3 103.4 182.0 299.2 518.0 678.7 285.8 309.3

1800 78.5 64.2 211.2 365.5 626.9 746.7 325.2 514.7

2200 75.1 58.1 261.7 426.8 714.7 805.9 681.8 1044.8

2600 72.0 51.2 365.8 662.7 827.0 826.4 1904.9 2991.4

3000 99.9 112.8 519.7 1043.6 1265.3 909.5 8236.7 5676.6

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–2584 2573

For a randomly generated instance of a symmetric job shopscheduling problem, let the random variable T denote the executiontime needed to enumerate the allocation search space for thatproblem after allocation Wn has been found by LBSA and evaluatedby the Virtual Factory. Table 2 shows the sample mean T andstandard deviation ST of the observed values of T for a set of suchrandomly generated problems. There were 200 randomly generatedproblems solved for each job shop type and due-date range using agiven staffing level, but the sample statistics in Table 2 werecomputed using only the problems in which allocation Wn did notsatisfy either of the optimality criteria presented in Section 3 (thenumber of problems in each sample can be obtained using Table 1).

The results in Table 2 illustrate that the average time neededto enumerate the allocation search space makes it an impracticalstrategy for finding a VF-best allocation (which we are looking forbecause allocation Wn is not optimal). Similar results wereobtained for the asymmetric job shop. We now turn to heuristicsearch strategies in the search for a VF-best allocation.

4.2. Local neighborhood search strategies (LNSS)

Recall that the Lower Bound Search Algorithm (LBSA) terminates ifeither of two termination conditions is satisfied as discussed at theend of Section 2 and in a remark in the Online Supplement. Twodifferent search strategies are developed in Sections 4.2.1 and 4.2.2,based on which condition is satisfied when LBSA terminates. Bothsearch strategies evaluate allocations in what can be termed the local

neighborhood of allocation Wn, and together they constitute what isreferred to as the Local Neighborhood Search Strategy (LNSS).

4.2.1. Type-a allocations

If the LBSA terminates because Condition 1 is satisfied, then noadditional workers can be assigned to machine group k, the currentlyselected constraining machine group of allocation Wn. Since nothingfurther can be done to decrease the lower bound on Lmax for machinegroup k, we consider additional worker allocations fWag derived fromWn that satisfy the condition LBWa ¼ LBWn , and that are called type-aallocations. Algorithm 1 is designed find a type-a allocation Wa thathas one of the following properties:

1.

Wa is both an optimal and a VF-best allocation; or 2. VFWa is the smallest value of Lmax delivered by the Virtual

Factory taken over all type-a worker allocations encounteredduring the execution of Algorithm 1.

A type-a allocation is found by allocating workers to minimize thelower bound on Lmax of the machine group i with the largest valueof LBmgi

, ignoring any machine groups that satisfy Condition 1 ofLBSA since the respective lower bounds on Lmax for these machinegroups cannot be improved. Any machine group i that has a full

complement of 9Mi9 workers assigned to it may not donate aworker, based on the assumption that unassigning a worker fromsuch a machine group would cause it to become constraining. Inaddition, once a machine group j donates a worker to anothermachine group, machine group j may not receive an additionalworker. Similarly, once a machine group j receives a worker fromanother machine group, machine group j may not donate a worker.Algorithm 1 terminates when one of the following conditions occurs:(i) the maximum search time has elapsed; (ii) a type-a allocation Wa

is encountered that satisfies VFWa ¼ LBWn so that Wa is both an optimaland a VF-best allocation; or (iii) the type-a allocation Wa with thesmallest possible value of VFWa has been found.

Algorithm 1. Algorithm to find Type-a allocations.

Wa’Wn

VFWa’VFWn

For each jAf1, . . . ,Sg

If wjðWnÞ ¼ 9Mj9

MayDonate½j�’false

MayReceive½j�’false

else if wjðWnÞ ¼ 1

MayDonate½j�’false

MayReceive½j�’true

elseMayDonate½j�’true

MayReceive½j�’true

l’0

Wl’Wn

While ðSearchTimeElapsedo45 secÞ

For each jAf1, . . . ,Sg

If MayDonate½j� ¼ true and LBmgjðwjðW

lÞ�1ÞrLBWn

If there exists mgi such that mgiamgj and

MayReceive½i� ¼ true

Wlþ1’Wl

wiðWlþ1Þ’wiðW

lÞþ1

wjðWlþ1Þ’wjðW

l�1

l’lþ1

MayDonate½i�’false

MayReceive½j�’false

If wjðWlÞ ¼ 1

MayDonate½j�’false

If wiðWlÞ ¼ 9Mi9

MayReceive½i�’false

Evaluate Wl using the Virtual Factory.If VFWl oVFWa

Wa’Wl

VFWa’VFWl

If VFWl ¼ LBWn

Deliver the optimal allocation Wa with VF-bestvalue VFWa and stop.

Deliver the type-a allocation Wa with the smallest possiblevalue VFWa and stop.

4.2.2. Type-b allocations

If LBSA terminates at allocation Wn with constraining machinegroup kAKðWn

Þ because Condition 2 is satisfied, then the neigh-borhood N ðWn,kÞ of Wn defined by Eq. (4) consists of up to S�1allocations, where for jAf1, . . . ,Sg\fkg, the jth neighbor fj ofWn is defined in terms of Wn by the worker reassignmentwjðf

jÞ ¼wjðW

nÞ�1 and wkðf

jÞ ¼wkðW

nÞþ1, with wlðf

jÞ ¼wlðW

nÞ

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–25842574

for la j,k (assuming that it is feasible to obtain fj from Wn byreassigning one worker from mgj to mgk; otherwise fj does notexist). For each allocation fjAN ðWn,kÞ, we define that allocation’sreduced neighborhood

Oðfj,Wn,kÞ � fju

fju is a neighbor of fjAN ðWn,kÞ with wkðfjuÞ

¼wkðfjÞ�1 and wuðf

juÞ ¼wuðf

jÞþ1 for some

u=2fj,kg; and wlðfjuÞ ¼wlðf

jÞ for l=2fk,ug:

��������8>><>>:

9>>=>>;:ð7Þ

In other words if allocation fj exists, then for uAf1, . . . ,Sg\fj,kg,the uth neighbor fju of allocation fj is defined by the workerreassignment wkðf

juÞ ¼wkðf

jÞ�1 and wuðf

juÞ ¼wuðf

jÞþ1, with

wlðfjuÞ ¼wlðf

jÞ for lak,u (assuming that it is feasible to obtain

fju from fj by reassigning one worker from mgk to mgu;otherwise, fju does not exist).

The worker allocations defined by the right-hand side ofEq. (7) are called type-b allocations; and associated with Wn andthe currently constraining machine group k, we have the full setof type-b allocations

BðWn,kÞ �[

fj AN ðWn ,kÞ

Oðfj,Wn,kÞ:

For each type-b allocation fju that belongs to the reducedneighborhood Oðfj,Wn,kÞ of the allocation fjAN ðWn,kÞ, we have

LBfju ¼ LBfj ¼ LBmgjðwjðf

jÞÞ ¼ LBmgj

ðwjðWnÞ�1Þ;

and thus allocation fj and all the type-b allocations in its reducedneighborhood Oðfj,Wn,kÞ are equally promising candidates to beevaluated by the Virtual Factory. Our search strategy is toevaluate each neighbor of allocation Wn and the type-b allocationsassociated with that neighbor using the Virtual Factory, where theevaluations are performed in order of increasing value of thelower bound on Lmax for the candidate allocations. The evalua-tions continue until we find an optimal allocation, or themaximum search time has elapsed; and finally we deliver theallocation Wb whose associated value VFWb is the smallest VF-valuetaken over all the allocations in N ðWn,kÞ [ BðWn,kÞ that wereevaluated by the Virtual Factory.

Remark 6. Depending on whether LBSA terminates by satisfyingeither Condition 1 or Condition 2, LNSS respectively deliverseither Wa or Wb, the best allocation of the relevant type that wasencountered in the execution of LNSS.

4.3. Queuing time search strategies

In the course of searching for the best schedule given an allocationW, the Virtual Factory computes the estimated queuing time of eachjob at each machine visited by that job. For the job with maximallateness Lmax (henceforth simply called the Lmax job), the VirtualFactory computes the job’s estimated queuing time in a machinegroup by summing the estimated queuing times for that job at eachmachine visited by the job on its route through the machine group.We use these estimated machine-group queuing times to identify themachine groups for which the assignment of an additional workermay reduce Lmax, as well as the machine groups for which theunassignment of a worker will probably not increase Lmax. Note thatboth queuing time search strategies presented here require an initial‘‘seed’’ allocation whose estimated queuing times can be used todetermine the next allocation searched.

4.3.1. Queuing time search strategy 1 (QSS1)

In the first queuing time search strategy (referred to as QSS1),after an allocation has been evaluated, the estimated machine-group queuing times for the Lmax job are used to determine a new

allocation as follows. Since each machine group must have atleast one worker assigned to it, there are only W�S workers thatmust be allocated. Let Proci denote the sum of the processingtimes of all jobs that must be processed on the machines inmachine group i, and let Procjs denote the sum of the processingtimes of all the jobs that must be processed in the job shop.Let LmaxQ i denote the estimated queuing time in machine groupi for the Lmax job, and let LmaxQ

jsdenote the estimated total

queuing time in the job shop for the Lmax job. Then the additionalnumber of workers that will be allocated to machine group i is

minProciþLmaxQ i

ProcjsþLmaxQjs

� �� ðW�SÞ

� �,9Mi9�1

� �: ð8Þ

Because of the rounding down in Eq. (8) and the constraint on thetotal number of workers to be assigned, there may be workersthat still need to be assigned after the S machine groups havebeen allocated workers using Eq. (8). In this case, the remainingworkers are assigned sequentially to the machine groups with thelargest values of the remainder

ProciþLmaxQ i

ProcjsþLmaxQjs

� �� ðW�SÞ

� ��

ProciþLmaxQ i

ProcjsþLmaxQjs

� �� ðW�SÞ

� �,

ð9Þ

that do not already have 9Mi9 workers assigned to them, wherei¼1,y,S. If the resulting allocation Wq1 has not already beenevaluated, then it is evaluated using the Virtual Factory and newestimated queuing times are obtained. The new estimated queu-ing times are then used in Eqs. (8) and (9) to obtain a newcandidate allocation; and this process is repeated until one of thefollowing termination conditions is satisfied: (i) the currentallocation Wq1 has already been evaluated; (ii) the current alloca-tion Wq1 satisfies the optimality criterion VFWq1 ¼ LBWn ; or (iii) themaximum search time has elapsed. When QSS1 terminates, itdelivers the allocation Wq1 whose associated value VFWq1 is thesmallest such VF-value taken over all allocations evaluated by theVirtual Factory during the course of executing QSS1.

4.3.2. Queuing time search strategy 2 (QSS2)

The second queuing time search strategy (referred to as QSS2)uses the estimated queuing time information to modify the base

allocation (defined as the allocation whose associated VF-value isthe smallest encountered so far) by selecting a machine group tohave one worker unassigned from it, and selecting a differentmachine group to have a worker reassigned to it. The receiving

machine group i is the machine group that lies on the Lmax job’sroute, has the largest LmaxQi value, and can receive a worker(i.e., the machine group i currently has fewer workers assigned toit than machines in the group). The donating machine group j is themachine group that does not lie on the Lmax job’s route, has thesmallest Procj value, and can donate a worker (i.e., it currently hasmore than one worker assigned to it). The motivation behind thisapproach to modifying the base allocation is that the receivingmachine group appears to have the greatest need for one addi-tional worker, while the donating machine group appears to bemost likely to perform at an acceptable level after the loss of oneworker.

For the modified allocation Wq2, if the associated value VFWq2 isless than the smallest VF value found so far, then Wq2 becomes thebase allocation. On the other hand if VFWq2 exceeds the smallest VFvalue found so far, then Wq2 is discarded, the base allocation remainsunchanged, and a new allocation is derived from the base allocationby selecting a new donating machine group and a new receivingmachine group. For a given base allocation, each machine group canbe either a donating or receiving machine group at most once.Iterations of QSS2 continue until one of the following terminationconditions is satisfied: (i) the current modified allocation Wq2 has

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–2584 2575

already been evaluated; (ii) the current modified allocation Wq2

satisfies the optimality criterion VFWq2 ¼ LBWn ; or (iii) the maximumsearch time has elapsed. When QSS2 terminates, it delivers theallocation Wq2 whose associated value VFWq2 is the smallest suchVF-value taken over all allocations evaluated by the Virtual Factoryduring the course of executing QSS2.

4.4. Combined heuristic search procedure

The three heuristic search strategies are combined into a singleheuristic search procedure (HSP) as follows. Procedure HSP startswith LNSS, which is initialized using the ‘‘seed’’ allocation Wn.When LNSS terminates, QSS1 is initialized using as the ‘‘seed’’allocation either Wa or Wb as returned by LNSS. When QSS1terminates, QSS2 is initialized using the ‘‘seed’’ allocation Wq1

returned by QSS1. When QSS2 terminates, it returns allocationWq2. After all stages of the combined heuristic search procedureare complete, HSP delivers WHSP, the allocation yielding thesmallest VF-value taken over all allocations encountered duringthe course of executing HSP. Alternative heuristic search proce-dures were tried (e.g., using allocation Wn as the ‘‘seed’’ allocationfor the queuing time search strategies and changing the order ofexecuting the three search strategies); however the version ofHSP presented here was found to provide the best results overall.

5. Experimentation

5.1. Experimental design

Experiments were performed to explore how HSP performedin different types of DRC job shops, with different staffing levels,and over a variety of due-date ranges. The experimental job shopused here is the same as that used by Lobo et al. [11]. The job shophas 80 machines split into 10 machine groups, where eachmachine group has eight machines. There are 1200 jobs thatrequire processing. The due-date of each job is a function of thedue-date range, which varies from 200 to 3000 in increments of400. (Preliminary experiments showed that, when the maximumdue-date exceeded 3000, the maximum flow time in the job shopwas less than the greatest due-date, an unrealistic scenario.) Theprocessing time of each operation is a random variable sampledfrom the discrete uniform distribution on the integers f1,2, . . . ,40g; Lobo et al. [11] found that job shop performance wasinsensitive to this assumption. Every job has between 6 and 10operations, and at most 3 of those operations can occur in thesame machine group (not necessarily consecutively).

Demirkol et al. [20] use the parameters of due-date range and theexpected number of tardy jobs to determine job due-dates forbenchmark job shop problems in which Lmax is the objective to beminimized. However, Hodgson et al. [14] find that if the due-date‘‘range is held constant, the tightness of the due-dates does notaffect the optimal sequence.’’ Thus in this article the difficulty ofproblems being solved is varied by varying the due-date range. Fourdifferent overall job shop staffing levels are considered, namely, 60%,70%, 80%, and 90% (a 90% staffing level corresponds to there being 72workers available for allocation to the 80 machines).

The type of job shop, reflected in the symmetric (balanced) andasymmetric (unbalanced) loading of the machine groups, alsoforms part of the experimental design. In the symmetric version,every machine group has an equal probability of being on a job’sroute, which results in each of the machine groups seeing 10% ofthe total workload on average. In the asymmetric version, somemachine groups have a greater probability of being on a job’sroute than others, so that four machine groups each see morethan 10% of the total workload on average, while the other six

machine groups each see less than 10% of the total workload onaverage (see Lobo et al. [11] for the exact percentages).

The combination of the job shop type, the staffing level, andthe due-date range of the jobs defines a class of DRC job shopscheduling problems. For each class of problems, 200 randomlygenerated problem instances were solved; and for i¼ 1, . . . ,200,the ith randomly generated problem instance is the same probleminstance that is used to compare the performance of the alloca-tions WHSP and Wn at each staffing level. (Appendix A of Lobo et al.[11] contains a detailed discussion of how the problem instancesare randomly generated). Using randomly generated problems inthis manner allows us to make sharper comparisons between theVirtual Factory’s performance using allocation WHSP versus itsperformance using allocation Wn; moreover, this performancecomparison extends to the full range of staffing levels [21].

The maximum time allowed for searching was 60 s. In the realworld, the allocation of workers in the job shop would typicallyoccur no more frequently than once per shift; thus it wasconsidered reasonable to spend up to one minute on searchingfor allocation WVFB. All experimentation was performed on com-puters that had an Intel Xeon W3520 processor running at2.67 GHz with 4 GB of RAM.

5.2. Evaluation of HSP

For a given problem instance, if allocation Wn does not satisfyeither of the optimality criteria, then the value VFWVFB representsthe smallest Lmax value that can be attained using the VirtualFactory to evaluate all feasible allocations. As a result, the qualityof allocation WHSP was measured by comparing VFWHSP with VFWVFB ,and not with LBWn , the lower bound on Lmax. The performance ofallocation WHSP relative to the VF-best allocation is defined as thedifference VFWHSP�VFWVFB . The average difference VFWn�VFWVFB isalso given to illustrate the improvement in the average Lmax valuethat can result from the use of HSP.

For a given class of DRC job shop scheduling problems, we canquantify the improvement in the average VF-value delivered byHSP using a one-sided skewness-adjusted 99% confidence intervalfor the expected value of the paired difference VFWn�VFWHSP thatresults from applying LBSA and HSP to the same randomlygenerated problem instance (from the given problem class) asexplained in the fourth paragraph of Section 5.1. If the confidenceinterval for the mean difference excluded the value 0, then weconcluded that using HSP yielded a statistically significantimprovement in the average VF-value compared with the alter-native strategy of merely using allocation Wn. Section A2 of theOnline Supplement provides a detailed procedure for calculatingskewness-adjusted confidence intervals for the expected value ofpaired differences of the form D¼ VFW1�VFW2 . Owing to thelimited space available, only selected skewness-adjusted confi-dence intervals are presented in this article. The reader is referredto Section A3 of the Online Supplement which contains allskewness-adjusted confidence intervals for the expected differ-ences of the form E½VFWn�VFWHSP �.

If allocation Wn satisfies either of the optimality criteria ofSection 3, then it is not necessary to apply HSP to the problem athand. Thus, for a given problem type using a particular staffinglevel, not all of the Q¼200 randomly generated problemsrequired the application of HSP. Because of this, for a given typeof job shop, staffing level, and due-date range, the average valuesof the differences VFWn�VFWVFB and VFWHSP�VFWVFB are not necessa-rily averages computed over 200 observations. In the tablescontaining the skewness-adjusted 99% confidence intervals forthe mean of the paired differences, Q 0 is the number of randomlygenerated problems (out of 200) for which allocation Wn did notsatisfy the hypotheses of either Theorem 1 or Theorem 2. From

Fig. 1. Average deviation of VFWHSP and VFWn from VFWVFB , symmetric job shop.

Table 3Skewness-adjusted 99% C.I. on the mean of the paired difference D¼VFWn�VFWHSP

(expressed in units of average operation processing time) in the symmetric job

shop with 60% staffing.

Due-date range Q 0 D SD cSkD99% Confidence interval

200 65 0.377 0.560 1.860 ½0:237,1Þ

600 55 0.388 0.660 1.959 ½0:212,1Þ

1000 50 0.326 0.567 2.287 ½0:172,1Þ

1400 51 0.422 0.780 2.740 ½0:218,1Þ

1800 50 0.325 0.645 2.483 ½0:152,1Þ

2200 47 0.388 0.701 2.645 ½0:198,1Þ

2600 54 0.260 0.543 3.263 ½0:127,1Þ

3000 65 0.182 0.436 3.439 ½0:084,1Þ

Table 4Skewness-adjusted 99% C.I. on the mean of the paired difference D¼ VFWn�VFWHSP

(expressed in units of average operation processing time) in the symmetric job

shop with 70% staffing.

Due-date range Q 0 D SD cSkD99% Confidence interval

200 104 0.515 0.873 1.905 ½0:338,1Þ

600 105 0.432 0.814 2.607 ½0:275,1Þ

1000 105 0.462 0.766 2.186 ½0:311,1Þ

1400 117 0.451 0.858 3.342 ½0:299,1Þ

1800 123 0.412 0.884 3.732 ½0:262,1Þ

2200 135 0.361 0.790 3.126 ½0:228,1Þ

2600 146 0.443 0.850 2.486 ½0:301,1Þ

3000 160 0.456 0.858 2.495 ½0:318,1Þ

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–25842576

the random sample fDi : i¼ 1, . . . ,Q 0g of observed differences, wecomputed the sample mean D, the sample variance S2

D, and thesample skewness cSkD. Formulas for calculating each of thesestatistics can be found in Section A2 of the Online Supplement.

5.2.1. Symmetric job shop

For the symmetric job shop, Fig. 1 shows the average perfor-mance of allocations WHSP and Wn when each allocation wascompared with allocation WVFB. Tables 3 and 4 contain one-sidedskewness-adjusted 99% confidence intervals for the mean of thepaired difference VFWn�VFWHSP when the staffing levels were 60%and 70%, respectively. All confidence intervals (see Section A3 of

the Online Supplement) indicated that for all staffing levels in asymmetric job shop, there was statistically significant improve-ment (i.e., a reduction) in the average value of Lmax as a result ofusing the allocation WHSP delivered by HSP compared with thealternative of simply using allocation Wn.

5.2.2. Asymmetric job shop

For the asymmetric job shop, Fig. 2 shows the average perfor-mance of allocation Wn and allocation WHSP when compared withallocation WVFB. The one-sided skewness-adjusted 99% confidenceintervals for the mean of the paired difference VFWn�VFWHSP in the caseof 80% and 90% staffing are given in Tables 5 and 6, respectively. All

Fig. 2. Average deviation of VFWHSP and VFWn from VFWVFB , asymmetric job shop.

Table 5Skewness-adjusted 99% C.I. on the mean of the paired difference D¼ VFWn�VFWHSP

(expressed in units of average operation processing time) in the asymmetric job shop

with 80% staffing.

Due-date range Q 0 D SD cSkD99% Confidenceinterval

200 1 0.000 – – –600 5 0.000 0.000 – –

1000 15 0.107 0.205 1.814 ½0:006,1Þ1400 25 0.051 0.131 2.847 ½0:005,1Þ1800 27 0.204 0.354 1.975 ½0:074,1Þ2200 38 0.182 0.302 1.969 ½0:087,1Þ2600 49 0.167 0.257 2.006 ½0:095,1Þ3000 60 0.149 0.186 1.470 ½0:099,1Þ

Table 6Skewness-adjusted 99% C.I. on the mean of the paired difference D¼VFWn�VFWHSP

(expressed in units of average operation processing time) in the asymmetric job

shop with 90% staffing.

Due-date range Q 0 D SD cSkD99% Confidence

interval

200 1 0.000 – – –

600 5 0.000 0.000 – –

1000 16 0.155 0.232 1.042 ½0:031,1Þ

1400 26 0.060 0.130 2.564 ½0:014,1Þ

1800 27 0.172 0.280 2.067 ½0:069,1Þ

2200 36 0.172 0.270 1.976 ½0:085,1Þ

2600 48 0.120 0.238 2.608 ½0:056,1Þ

3000 60 0.186 0.219 1.180 ½0:126,1Þ

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–2584 2577

confidence intervals (see Section A3 of the Online Supplement)indicated that for all staffing levels in an asymmetric job shop, therewas statistically significant improvement (reduction) in the averagevalue of Lmax as a result of using the allocation WHSP delivered by HSPcompared with the alternative of simply using allocation Wn.

5.2.3. Overall results

The results indicated that HSP was effective for all classes ofDRC job shop scheduling problems considered. This effectivenesswas most apparent in the asymmetric job shop with 60% staffing,and the symmetric job shop with 60% and 70% staffing, wherea VF-best allocation was found nearly 100% of the time; and ina large percentage of such problem instances, we also observedthat LBSA terminated because Condition 2 was satisfied (seeTable 8 of Lobo et al. [11]). HSP was also very effective in theasymmetric job shop with 70% staffing, where the reduction inthe average Lmax value was approximately 2.30 average operationprocessing times. The effectiveness of HSP appeared to diminishslightly as the staffing level increased, and this trend was mostnoticeable in the symmetric job shop.

The effectiveness of HSP and the trend can be explained byconsidering the following observations (fully explained in Section5.2.3. of Lobo et al. [11]):

1.

If LBSA terminated because Condition 2 was satisfied, then theallocations W surrounding allocation Wn satisfied LBWZLBWn ,and usually satisfied LBW4LBWn .

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–25842578

2.

If LBSA terminated because Condition 1 was satisfied, thenthere were a large number of allocations W surroundingallocation Wn for which LBW ¼ LBWn .

3.

As the staffing level increased, termination Condition 2 of LBSAoccurred less frequently.

As described in Section 4.1, the partial enumeration strategyuses the initial requirement LBWoVFWn to decide whether anallocation W merits evaluation with the Virtual Factory. In light ofthe first observation, when Condition 2 was satisfied, there were,in general, relatively few allocations satisfying this initial require-ment. In light of the second observation, when Condition 1 wassatisfied, there were, in general, a large number of allocationssatisfying this initial requirement. Therefore when Condition2 was satisfied, the majority of allocations searched by the partialenumeration strategy were also searched by HSP. However, thiswas not the case when Condition 1 was satisfied. Combined withthe third observation, this explains why HSP was most effectivewhen the staffing level was lower, and why HSP’s effectivenessdiminished as the staffing level increased.

If allocation WHSP does not satisfy the optimality criterion ofTheorem 1 (i.e., if VFWHSP aLBWn ), then the value VFWHSP is an upperbound on Lmax for the JM9W9Lmax DRC job shop schedulingproblem. Given that allocation Wn did not satisfy either optimalitycriteria presented in Section 3, and allocation WHSP did not satisfythe optimality criterion of Theorem 1, it is important to have ameans of assessing the quality of allocation WHSP, and in turn, therelative quality of the upper bound on Lmax. The following sectionprovides a probabilistic approach to making such an assessment.

6. A probabilistic approach to finding a VF-best allocation

If allocation Wn does not satisfy either of the optimality criteriaoutlined in Section 3, and if allocation WHSP does not satisfy theoptimality criterion of Theorem 1, then the following questionarises: is the allocation WHSP delivered by HSP in fact a VF-bestallocation? Enumeration of the allocation search space is guaran-teed to yield a VF-best allocation, while heuristics (such as HSP)are not guaranteed to yield even a VF-best allocation. Therefore inthe context of a specific DRC job shop scheduling problem(subsequently called the ‘‘designated problem’’), it is importantto assess the quality (i.e., closeness to optimality) of an allocationthat has been found by LBSA, HSP, or some other procedure; inparticular, we need to assess our degree of confidence that thecurrent allocation is a VF-best allocation for the designatedproblem. The theoretical probability distributions developed inthe next section are specifically designed to enable such anassessment. Section 6.1 details a method for estimating a prob-ability distribution that characterizes the performance of aVF-best allocation relative to LBWn by taking into account allpossible independent simulated replications of the designatedproblem. In the following discussion, the result of each simulatedreplication of the designated problem will be called a ‘‘simulatedproblem based on the designated problem.’’ Section 6.2 sum-marizes the results of fitting similar probability distributions forseven different classes of DRC job shop scheduling problems.

6.1. Estimating the probability that W is a VF-best allocation

6.1.1. An empirical distribution describing performance of allocation

W relative to LBWn

To gauge the likelihood that a specific (fixed) allocation W isVF-best for the designated DRC job shop scheduling problem, wegenerate Q simulated problems based on the designated problemusing the method detailed in Appendix A of Lobo et al. [11]. In

particular each simulated problem is identical to the designatedproblem in the following key respects:

�

It has the same number of jobs; � It has the same number of machines and the same number

of machine groups;

� It has the same pattern of symmetric or asymmetric loading

of the machine groups;

� It has the same level of staffing—i.e., the ratio of the number

of workers to the number of machines expressed as a percen-tage between 0% and 100%; and

� It has the same due-date range.

On the other hand for each simulated problem, the followingcharacteristics of that problem are randomly sampled fromappropriate probability distributions:

�

the due-date of each job; � the number of operations required to complete each job; � the route of each job through the job shop (i.e., the ordered

sequence of machines that the job must visit); and

� the processing time of each operation.

After generating Q simulated problems based on the desig-nated problem, we restrict our attention to only the Q 0 simulatedproblems in which the LBSA-delivered allocation Wn did notsatisfy either of the optimality criteria presented in Section 3,where of course Q 0rQ . The performance of an allocation Wrelative to the lower bound on Lmax, denoted by PLBðWÞ, is definedas the difference between the following: (a) the value of Lmax forthe schedule generated by the Virtual Factory using allocation W;and (b) the lower bound on Lmax for the LBSA-delivered allocationWn. Therefore we take

PLBðWÞ �VFW�LBWn : ð10Þ

For the ith simulated problem (i¼ 1, . . . ,Q 0Þ whose correspondingLBSA-delivered allocation Wn

i does not satisfy the optimalitycriteria of Section 3, we determine a VF-best allocation usingthe partial enumeration strategy presented in Section 4.1. Ahistogram can be constructed from the resulting data set com-prising the Q 0 values of PLBðWVFB

Þ, that is

fPLBðWVFBi Þ : i¼ 1, . . . ,Q 0g: ð11Þ

The histogram’s horizontal axis, which represents the range ofpossible values of the random variable PLBðWVFB

Þ, is partitionedinto equal-width bins (class intervals) of PLBðWVFB

Þ values, whereScott’s rule [22] is used to determine the bin width and thenumber of bins in the histogram. The histogram’s vertical axisrepresents the likelihood with which PLBðWVFB

Þ values for simu-lated problems fall in the corresponding bins; therefore for eachbin, the area of the rectangle sitting over that bin is equal to therelative frequency with which the simulation-generated PLBðWVFB

Þ

values fall in that bin. Sections 6.1.2 and 6.3 that follow bothcontain examples of the histogram and the associated empiricalcumulative distribution function (cdf) describing the data set (11)arising from a set of simulated problems that are based on adesignated DRC job shop scheduling problem. An extensivecollection of the aforementioned graphs can also be found inLobo et al. [23].

Remark 7. In practice even for a medium-size DRC job shopscheduling problem and for a moderate value of Q 0, it can becomputationally prohibitive to determine a VF-best allocation

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–2584 2579

for each simulated problem using the partial enumeration strat-egy presented in Section 4.1. We are currently developingcomputationally practical methods to generate random samplesfrom a sufficiently close approximation to the desired distributionof PLBðWÞ. Although in this section we use the ‘‘ideal’’ approach togenerating the data set (11) based on the partial enumerationstrategy for finding a VF-best allocation, this ideal approach isintended mainly to demonstrate the potential of our method forestimating the probability (or the user’s degree of confidence)that a specific allocation W delivered by HSP for a designated DRCjob shop scheduling problem is a VF-best allocation for thatproblem.

6.1.2. A theoretical probability distribution describing PLBðWVFBÞ

Given a designated DRC job shop scheduling problem togetherwith a specific allocation W of workers to machines for thatproblem (where W is perhaps the allocation WHSP delivered byHSP) such that W does not satisfy the optimality criteria of Section3, we seek to estimate our level of confidence that W is a VF-bestallocation for the designated problem. If W is in fact a VF-bestallocation, then we may regard the difference PLBðWÞ ¼VFW�LBWn

defined by Eq. (10) as another observation from the population ofdifferences of the form

PLBðWVFBÞ ¼ VFWVFB�LBWn , ð12Þ

that encompasses all possible simulated problems based on thedesignated problem; and we will estimate the cdf of this popula-tion using the simulation-generated random sample (11). In thespirit of statistical hypothesis testing, we may therefore test thenull hypothesis that W is a VF-best allocation for the designatedproblem by estimating the probability that for a simulatedproblem based on the designated problem, the associated randomvariable PLBðWVFB

Þ in Eq. (12) will be greater than the particular(fixed) value PLBðWÞ for the designated problem. This upper tailprobability can be viewed as the p-value (significance probability)for the test of the null hypothesis that W is a VF-best allocation forthe designated problem [24, pp. 221–223]. From a differentperspective, we can interpret this upper tail probability as ourlevel of confidence that W is a VF-best allocation for the desig-nated problem. For example, if approximately 95% of the observa-tions in the data set (11) are larger than PLBðWÞ for the designatedproblem, then we can conclude that allocation W is better thanapproximately 95% of the VF-best allocations for simulatedproblems based on the designated problem; and thus we can be95% confident that allocation W is in fact a VF-best allocation forthe designated problem.

6.2. Fitting a theoretical distribution to random samples of PLBðWVFBÞ

To facilitate their use in practice, we seek to approximate thehistogram and the empirical cdf based on the data set (11) for agiven problem class by fitting an appropriate standard probabilitydistribution to that data set. We used the Stat::Fit software [25]for this purpose, which provided us with the following graphs:

�

A graph of the histogram of the data set (11) superimposed onthe fitted probability density function (pdf); and � A graph of the empirical cdf of the data set (11) superimposed

on the fitted cdf

In addition, a p-value for the chi-squared goodness-of-fit test wasprovided for each fitted distribution.

Because Q 0 varied substantially across the different problemclasses, care was taken when using the p-values as indicators of thegoodness-of-fit. When Q 0 is very large, ‘‘practically insignificant

discrepancies between the empirical and theoretical distributionsoften appear statistically significant’’ [26]. On the other hand, verysmall values of Q 0 often result in relatively large p-values becausestandard goodness-of-fit statistics have low power to distinguishbetween different distributions based on small samples. Theseconsiderations indicate that the p-value cannot be relied on as thesole goodness-of-fit criterion. The p-value from the chi-squaredgoodness-of-fit test and graphs of both the fitted pdf and the fittedcdf were used to decide which distribution best characterized agiven data set.

If the data set (11) could be adequately modeled by a standardcontinuous distribution, then we fitted the associated pdf to thedata set as detailed in Section 6.2.1 below. However, in somesituations, the data set (11) exhibited a substantial percentageof observations at zero. In those situations, we used a mixeddistribution with nonzero probability mass at the origin andwith a continuous right-hand tail having a standard functionalform as described in Section 6.2.2 below. In other situationswe were forced to use a discrete probability mass function (pmf)to describe the data set (11) as explained in Section 6.2.3.The latter situation arose in cases where there were relativelyfew distinct nonzero values of PLBðWVFB

Þ in the data set. Inthe fits that follow, the number of randomly generated probleminstances for each class of DRC job shop scheduling problem isQ¼500.

6.2.1. Continuous distributions

For the data set (11) associated with each class of DRC job shopscheduling problems, Stat::Fit was initially used to seek the best-fitting continuous distribution. When adequate fits were obtained,the following distributions were used:

�

the generalized Beta distribution, denoted as Betaða,b,a,bÞ,where a and b are the two shape parameters, a is the lowerlimit, and b is the upper limit; � the shifted Gamma distribution, denoted as Gammaða,b,aÞ,

where a is the shape parameter, b is the scale parameter,and a is the lower limit;

� the bounded Johnson distribution, denoted as Johnson-

SBðg,d,l,xÞ, where g and d are shape parameters, l is the scaleparameter representing the range of possible values of thecorresponding random variable, and x is the location para-meter representing the lower limit of the distribution [26];

� the unbounded Johnson distribution, denoted as Johnson-

SUðg,d,l,xÞ, where g and d are shape parameters, l is the scaleparameter (but not the range of possible values of thecorresponding random variable, which is infinite in bothdirections), and x is a scale parameter (but not the lower limitof the distribution, which is �1; see Kuhl et al. [26]);

� the shifted Lognormal distribution, denoted as Lognormalðm,s,aÞ,

where m and s are the mean and standard deviation of thecorresponding random variable respectively, and a is the lowerlimit; and

� the shifted (three-parameter) Weibull distribution, denoted as

Weibullða,l,aÞ, where a is the shape parameter, l is the scaleparameter, and a is the lower limit.

Fig. 3 depicts four examples of continuous distributions thatwere fitted to data sets of the form (11). Similar graphs and tablesthat summarize the fits for all problem classes fitted using acontinuous distribution can be found in Lobo et al. [23]. In thecorresponding graphs for data sets fitted with a continuousdistribution, the legend ‘‘Datapoints’’ identifies the size of thedata set, Q 0. Both the graphical evidence and the p-values for the

Fig. 3. Probability distribution fitting, continuous distribution fits.

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–25842580

goodness-of-fit tests indicated that in every problem class forwhich a continuous distribution was used to approximate thedata set, the resulting model was adequate.

6.2.2. Mixed distributions

If there was a nonnegligible probability mass at PLBðWVFBÞ ¼ 0,

then it was not possible to fit the data set with a standard

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–2584 2581

continuous distribution. In these cases, the data sets were fittedusing a mixed cdf of the form

FðxÞ ¼ p0F0ðxÞþð1�p0ÞFcðxÞ for �1oxo1,

where: (a) p0 is the proportion of observations of PLBðWVFBÞ that

are equal to zero; (b) F0ðxÞ is the cdf of the degenerate distributionwith unit probability mass at the origin so that

F0ðxÞ ¼0 for xo0,

1 for xZ0;

(

and (c) FcðxÞ is the conditional cdf of PLBðWVFBÞ given that

PLBðWVFBÞ40, where we may take FcðxÞ to be the cdf of any of

the distributions listed in Section 6.2.1.Two examples of problems fitted with a mixed distribution

are: (a) the asymmetric job shop with 70% staffing and a due-daterange of 1800; and (b) the symmetric job shop with 90% staffingand a due-date range of 2600. Fig. 4 displays the continuousdistribution that provided the ‘‘best fit’’ to the associated sub-sample consisting of the nonzero values of PLBðWVFB

Þ for each ofthe problem classes. The graphs in Fig. 4 also show Q 0ð1�p0Þ, thesize of the associated subsample (labeled ‘‘Datapoints’’), and thep-value for the chi-squared goodness-of-fit test. Tables thatsummarize the ‘‘best fits’’ for all problem classes fitted using amixed distribution, together with the associated graphical fits canbe found in Lobo et al. [23]. Both the graphical evidence and thep-values for the goodness-of-fit tests indicated that in everyproblem class for which a mixed distribution was used toapproximate the data set, the resulting model was adequate.

Fig. 4. Probability distribution fitting, par

6.2.3. Discrete distributions

For a number of problem types with given staffing levels (e.g.,the asymmetric job shop with a due-date range of 2200 and 90%staffing), there were so few PLBðWVFB

Þ values distinct from zero inthe data set that a continuous distribution could not be reliably fitto the remaining nonzero values. In this case, a discrete pmf wasdetermined for the data set. See Lobo et al. [23] for an example ofa discrete pmf fitting.

6.2.4. Fitting a distribution in general

The overall experimental design was composed of 64 differentclasses of DRC job shop scheduling problems (i.e., 64 differentcombinations of job shop type, staffing level, and due-date range).For each problem class, the data set of PLBðWVFB

Þ values wasconstructed as described in Section 6.1 with Q¼500. Due to thelimited space available, the graphs pertaining to the 64 differentcombinations cannot be presented in this article. The full set ofgraphs, along with an in depth description and critique of thefitting methodology, can be found in Lobo et al. [23].

More than 10% of the PLBðWVFBÞ values were equal to zero for

each of the following cases: (a) the symmetric job shop with 90%staffing and a due-date range of 200 through 2600; (b) theasymmetric job shop with 70% staffing; (c) the asymmetric jobshop with 80% staffing and a due-date range of 2600 and 3000; and(d) the asymmetric job shop with 90% staffing and a due-daterange of 2600 and 3000. In these cases, a conditional pdf was fittedto the data set composed of the nonzero PLBðWVFB

Þ values. Less than50 nonzero PLBðWVFB

Þ values were observed in each of the following

t Fcð�Þ of the mixed distribution fits.

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–25842582

cases: (i) the asymmetric job shop with 80% staffing and a due-daterange of 200 through 2200; and (ii) the asymmetric job shop with90% staffing and a due-date range of 200 through 2200. In cases(i) and (ii), we simply used the observed values in each data set todefine a discrete probability distribution.

We found that, in general, the generalized Beta distribution,the shifted Gamma distribution, the shifted Lognormal distribu-tion, and the shifted Weibull distribution could be used tocharacterize the majority of the data sets. The exception to thiswas the symmetric job shop with 80% staffing, where theunbounded Johnson distribution approximated the left-hand tailof the associated data sets substantially better than any of theother standard continuous distributions. (Kuhl et al. [26] discussseveral diverse engineering applications in which unboundedJohnson distributions yield superior fits to nonstandard tailbehavior.) In the symmetric job shop with 80% staffing and adue-date range of 3000 the small p-value was not a concern whenthe value of Q 0 and the graphs of the fitted pdf and cdf (see Fig. 3)were taken into account.

The practical applicability of our distribution-fitting approachis clearly demonstrated by the diversity of DRC job shop schedul-ing problems for which the associated data set (11) can beadequately modeled by these six different probability distribu-tions in a straightforward manner.

6.3. Using the theoretical probability distribution

This section illustrates the use of the theoretical probabilitydistribution through two examples, and demonstrates theimprovement in allocation quality that is achievable throughuse of HSP.

A random problem was generated for an asymmetric job shopwith 60% staffing and a due-date range of 1400. The fitted pdfcorresponding to such a combination of job shop type, staffinglevel, and due-date range is f 1ðxÞ �Weibullð1:36,55:42,11:49Þ(see Fig. 5). Allocation Wn did not satisfy either of the optimalitycriteria presented in Section 3, where LBWn ¼ 3162, VFWn ¼ 3313,and therefore PLBðWn

Þ ¼ 151.From this we know thatZ PLBðWn

Þ

0f 1ðxÞ dx¼ PrfPLBðWVFB

Þr151g

¼ 1�exp �151�11:49

55:42

� �1:36" #

¼ 0:9701,

that is, we are only 2.99% confident allocation Wn is a VF-best

Fig. 5. Probability distribution fitting, asymmetric

allocation. After using HSP, we found that PLBðWHSPÞ ¼ 44, and

thus we can conclude that

PrfPLBðWVFBÞrPLBðWHSP

Þg ¼ PrfPLBðWVFBÞr44g ¼ 0:3838,

indicating we are 61.62% confident that allocation WHSP is aVF-best allocation.

A random problem was generated for a symmetric job shopwith 80% staffing and a due-date range of 2200. The fitted pdfcorresponding to such a combination of job shop type, staffinglevel, and due-date range is f 2ðxÞ � Johnson-SUð�0:45,4:40,116:97,75:00Þ (see Fig. 6). Again, allocation Wn did not satisfyeither of the optimality criteria presented in Section 3, whereLBWn ¼ 1146, VFWn ¼ 1337, and therefore PLBðWn

Þ ¼ 191. We findthatZ PLBðWn

Þ

0f 2ðxÞ dx¼ PrfPLBðWVFB

Þr191g

¼F �0:45þ4:40 ln191�75:0

116:97

� �þ

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi191�75:0

116:97

� �2

þ1

s24 358<:9=;

¼ 0:9997,

(where Fð�Þ denotes the standard normal cdf), indicating we areonly 0.03% confident that Wn is a VF-best allocation. After usingHSP we found that PLBðWHSP

Þ ¼ 74, from which we can concludethat

PrfPLBðWVFBÞrPLBðWHSP

Þg ¼ PrfPLBðWVFBÞr74g ¼ 0:313,

indicating that we are 68.7% confident that WHSP is a VF-bestallocation.

Both examples illustrate that, when allocation Wn does notsatisfy the optimality criteria developed in Section 3, HSP is ableto find a significantly improved allocation WHSP. The theoreticalprobability distributions measure the quality of allocation WHSP byquantifying the uncertainty introduced by a heuristic searchmethod.

The analysis in this section was developed using the VirtualFactory as the heuristic scheduler. The theoretical basis for thisanalysis would be just as valid if another heuristic schedulerwere used.

7. Conclusions and recommendations

In this article we formulate and justify optimality criteria bywhich the allocation W that yields the smallest value of LBW can bedeclared optimal. In the case that allocation Wn fails to satisfyeither of the optimality criteria, we develop three different

job shop, 60% staffing, 1400 due-date range.

Fig. 6. Probability distribution fitting, symmetric job shop, 80% staffing, 2200 due-date range.

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–2584 2583

heuristic strategies that search for a VF-best allocation—i.e., anallocation that enables the Virtual Factory to generate a schedulewith the smallest possible value of Lmax that can be achieved byapplying the Virtual Factory to the problem at hand. The experi-mental results indicated that the search strategies, on the whole,provided a statistically significant improvement over the Lmax

value obtainable when using the LBSA-delivered allocation Wn,together with the Virtual Factory, to generate a schedule for thejob shop. We also present a simulation-based method for obtain-ing a probability distribution from which, for a designated DRCjob shop scheduling problem, we can estimate our degree ofconfidence that a specific allocation W is in fact a VF-bestallocation for the designated problem. This probability distribu-tion can be used to evaluate allocation quality, and quantify theuncertainty introduced through the use of a heuristic scheduler.As mentioned in Remark 7, rapid computational methods arerequired to obtain a simulation-based approximation to thisdistribution in practical problems of realistic size and complexity.Developing and implementing such methods is the primaryobjective of our ongoing research. Finally, even if an optimalallocation is not found, the value VFWHSP forms an upper bound onLmax for the JM9W9Lmax DRC job shop scheduling problem.

A promising direction for future work is to design astopping criterion for heuristic search strategies that makeseffective use of the probability distributions formulated in thisarticle. A good candidate for such a stopping criterion wouldbe LNSS, whose average execution time is the longest. Anotherdirection for future work is to extend the current research tohandle a DRC job shop in which workers have specific skills,and the workers may only be assigned to machine groupsmatching their skills. Although the latter system involves amore difficult worker-allocation problem, it is perhaps a moreaccurate representation of many DRC job shops encounteredin practice. We have good reason to believe that the frame-work developed in this article can be applied to such DRCjob shops.

Appendix A. Supplementary data

Supplementary data associated with this article can be found inthe online version at http://dx.doi.org/10.1016/j.cor.2013.02.008.

References

[1] Treleven M. A review of the dual resource constrained system research. IIETransactions 1989;21(3):279–87.

[2] Treleven M, Elvers D. An investigation of labor assignment rules in a dual-constrained job shop. Journal of Operations Management 1985;6(1):51–68.

[3] Lenstra J, Rinnooy Kan A. Computational complexity of discrete optimizationproblems. In: Hammer P, Johnson E, Korte B, editors. Discrete optimizationI:Proceedings of the advanced research institute on discrete optimizationand systems applications of the systems science panel of NATO and of thediscrete optimization symposium, Annals Of Discrete Mathematics, vol. 4.Elsevier;1979. p. 121–40.

[4] Nelson R. Labor and machine limited production systems. ManagementScience 1967;13(9):648–71.

[5] Gargeya V, Deane R. Scheduling research in multiple resource constrained jobshops: a review and critique. International Journal of Production Research1996;34(8):2077–97.

[6] Hottenstein M, Bowman S. Cross-training and worker flexibility: a review ofDRC system research. The Journal of High Technology Management Research1998;9(2):157–74.

[7] Jaber M, Neumann W. Modelling worker fatigue and recovery in dual-resourceconstrained systems. Computers & Industrial Engineering 2010;59(1):75–84.

[8] Kher H, Malhotra M, Philipoom P, Fry T. Modeling simultaneous workerlearning and forgetting in dual resource constrained systems. EuropeanJournal of Operational Research 1999;115(1):158–72.

[9] Kher H. Examination of flexibility acquisition policies in dual resourceconstrained job shops with simultaneous worker learning and forgettingeffects. The Journal of the Operational Research Society 2000;51(5):592–601.

[10] Malhotra M, Fry T, Kher H, Donohue J. The impact of learning and laborattrition on worker flexibility in dual resource constrained job shops.Decision Sciences 1993;24(3):641–64.

[11] Lobo B, Hodgson T, King R, Thoney K, Wilson J. An effective lower bound onLmax in a worker-constrained job shop. Computers & Operations Research2013;40(1):328–43.

[12] Hodgson T, Cormier D, Weintraub A, Zozom A. Satisfying due dates in largejob shops. Management Science 1998;44(10):1442–6.

[13] Vepsalainen A, Morton T. Improving local priority rules with global lead-timeestimates: a simulation study. Journal of Manufacturing & OperationsResearch 1988;1:102–18.

[14] Hodgson T, King R, Thoney K, Stanislaw N, Weintraub A, Zozom A. Onsatisfying due-dates in large job shops: idle time insertion. IIE Transactions2000;32(2):177–80.

[15] Hodgson T, Melendez B, Thoney K, Trainor T. The deployment schedulinganalysis tool (DSAT). Mathematical and Computer Modelling 2004;39(6–8):905–24.

[16] Thoney K, Hodgson T, King R, Taner M, Wilson A. Satisfying due-dates in largemulti-factory supply chains. IIE Transactions 2002;34(9):803–11.

[17] Zozom A, Hodgson T, King R, Weintraub A, Cormier D. Integrated job releaseand shop-floor scheduling to minimize WIP and meet due-dates. Interna-tional Journal of Production Research 2003;41(1):31–45.

[18] Weintraub A, Cormier D, Hodgson T, King R, Wilson J, Zozom A. Schedulingwith alternatives: a link between process planning and scheduling. IIETransactions 1999;31(11):1093–102.

[19] Schultz S, Hodgson T, King R, Thoney K. Minimizing Lmax for large-scale, job-shop scheduling problems. International Journal of Production Research2004;42(23):4893–907.

[20] Demirkol E, Mehta S, Uzsoy R. Benchmarks for shop scheduling problems.European Journal of Operational Research 1998;109(1):137–41.

[21] Chang YL, Sullivan R, Bagchi U, Wilson J. Experimental investigation of real-time scheduling in flexible manufacturing systems. Annals of OperationsResearch 1985;3(7):355–77.

[22] Scott D. On optimal and data-based histograms. Biometrika 1979;66(3):605–10.

B.J. Lobo et al. / Computers & Operations Research 40 (2013) 2569–25842584

[23] Lobo B, Thoney K, Hodgson T, King R, Wilson J. A probabilistic characterizationof allocation performance in a worker-constrained job shop. In: Uszoy R,Pulat S, Sarin S, editors. Essays in planning, scheduling and optimization: aFestschrift in Honor of Salah Elmaghraby. New York: Springer; forthcoming/http://people.engr.ncsu.edu/bjlobo/papers/loboetal_festschrift.pdfS.

[24] Bickel P, Doksum K. Mathematical statistics: basic ideas and selected topics.2nd ed Upper Saddle River, NJ: Pearson Prentice Hall; 2007.

[25] Geer Mountain Software Corp.. Stat::Fit, Version 2. South Kent, CT: GeerMountain Software Corp.; 2001.

[26] Kuhl M, Ivy J, Lada E, Steiger N, Wagner M, Wilson J. Univariate input models

for stochastic simulation. Journal of Simulation 2010;4:81–97.