deterministic scheduling krzysztof giaro and adam nadolski gdańsk university of technology

104
Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Upload: daniel-mathews

Post on 23-Dec-2015

220 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Deterministic Scheduling

Krzysztof Giaro and Adam NadolskiGdańsk University of Technology

Page 2: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Lecture Plan

1. Introduction to deterministic scheduling2. Critical path method3. Some discrete optimization problems4. Scheduling to minimize Cmax

5. Scheduling to minimize Ci

6. Scheduling to minimize Lmax

7. Scheduling to minimize the number of tardy tasks

8. Scheduling on dedicated processors

Page 3: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic Scheduling

Our aim is to schedule the given set of tasks (programs, etc.) on machines (or processors).

We have to construct a schedule that fulfils given constraints and minimizes the optimality criterion.

Deterministic model: All the parameters of the system and of the tasks are known in advance.

Genesis and practical motivation:• scheduling manufacturing processes,• project planning, • school or conference timetabling,• scheduling processes in multitask operational systems,• distributed computing.

Page 4: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic Scheduling

M 1

M 2

M 3

T 2

9

T 4 T 1

T 5 T 3

Graphical representation of the schedule – Gantt chart

Example 1. Five tasks with processing times p1,...,p5=6,9,4,1,4 have to be processed on three processors to minimize schedule length.

Why the above schedule is feasible?

General constraints in classical scheduling theory:• each task is processed by at most one processor at a time,

• each processor is capable of processing at most one task at a time,

• other constraints – to be discussed...

Page 5: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingProcessors characterization

Parallel processors (each processor is capable to process each task):

• identical processors – every processor is of the same speed,

• uniform processors – processors differ in their speed, but the speed does not depend on the task,• unrelated processors – the speed of the processor depend on the particular task processed.

M 1

M 2

M 3

T 2

9

T 4 T 1

T 5 T 3

Schedule on three parallel processors

Page 6: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingProcessors characterization

Dedicated processors Each job consists of the set of tasks preassigned to processors (job Jj consists of tasks Tij preassigned to Pi, of processing time pij). The job is completed at the time the latest task is completed, some jobs may not need all the processors (empty operations), no two tasks of the same job can be scheduled in parallel, a processor is capable to process at most one task at a time.

There are three models of scheduling on dedicated processors:• flow shop – all jobs have the same processing order through the machines coincident with machines numbering,• open shop – the sequence of tasks within each job is arbitrary ,• job shop – the machine sequence of each job is given and may differ between jobs.

Page 7: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingProcessors characterization

Dedicated processors – open shop (processor sequence is arbitrary for each job).

M 1

M 2

M 3

J 2

7

J 3

Z 1 J 1 J 2

J 2

J 1

J 1 J 3

J 3

Teachers

Classes

M1 M2 M3 J1 3 2 1J2 3 2 2J3 1 1 2

Example. One day school timetable.

Page 8: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingProcessors characterization

Dedicated processors – flow shop (processor sequence is the same for each job – task Tij must precede Tkj for i<k).

M 1

M 2

M 3

J b

10

J c

J a Z 1 J b

J b

J a

J a J c

J c

Robots

Details

M1 M2 M3 Ja 3 2 1Jb 3 2 2Jc 1 1 2

Ja Jb Jc

M1 M2 M3

Dedicated processors will be considered later ...

Example. Conveyor belt.

Page 9: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingTasks characterization

Task Tj : Processing time. It is independent of processor in the case of identical processors and is denoted pj. In the case of uniform processors, processor Mj speed is denoted bi and the processing time of Tj on Mj is pj/bj. In the case of unrelated processors the processing time of Tj on Mj is pij. Release (or arrival) time rj. The time at which the task is ready for processing. By default all release times are zero. Due date dj. Specifies a time limit by which Tj should be completed. Usually, penalty functions are defined in accordance with due dates, or dj denotes the ‘hard’ time limit (deadline) by which Tj must be completed.• Weight wj – expresses the relative urgency of Tj , by default wj=1.

There are given: the set of n tasks T={T1,...,Tn} and the set of m machines (processors) M={M1,...,Mm}.

Page 10: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingTasks characterization

Dependent tasks: In the task set there are some precedence constraints defined by a precedence relation. TiTj means that task Tj cannot be started until Ti is completed (e.g. Tj needs the results of Ti). In the case there are no precedence constraints, we say that the tasks are independent (by default). In the other case we say that the tasks are dependent. The precedence relation is usually represented as a directed graph in which nodes correspond to tasks and arcs represent precedence constraints (task-on-node graph). Transitive arcs are usually removed from precedence graph.

Page 11: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingTasks characterization

Example. A schedule for 10 dependent tasks (pj given in the nodes).

2

64T5 1

1

2 3 1T1 T2 T3 T4

T6

T7

T8

T9

T10

2

2

M 1

M 2

M 3

T 2

10

T 10 T5

Z 1 T 9 T 4

T 1

T 3

T 6

5

Z 1 T 7

T8

Page 12: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingTasks characterization

Example. A schedule for 10 dependent tasks (pj given in the nodes).

2

64T5 1

1

2 3 1T1 T2 T3 T4

T6

T7

T8

T9

T10

2

2

M 1

M 2

M 3

T 2

10

T 10 T5

Z 1 T 9 T 4

T 1

T 3

T 6

5

Z 1 T 7

T8

Page 13: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingTasks characterization

Example. A schedule for 10 dependent tasks (pj given in the nodes).

2

64T5 1

1

2 3 1T1 T2 T3 T4

T6

T7

T8

T9

T10

2

2

M 1

M 2

M 3

T 2

10

T 10 T5

Z 1 T 9 T 4

T 1

T 3

T 6

5

Z 1 T 7

T8

Page 14: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingTasks characterization

A schedule can be:• non-preemptive – preempting of any task is not allowed (default),• preemptive – each task may be preempted at any time and restarted later (even on a different processor) with no cost.

Preemptive schedule on parallel processors

M 1

M 2

M 3

T 2

9

T 1 Z 1

T 3

T 3

T 3

Page 15: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic Scheduling

Feasible schedule conditions (gathered): each processor is assigned to at most one task at a time, each task is processed by at most one machine at a time, task Tj is processed completely in the time interval [rj,), precedence constraints are satisfied, in the case of non-preemptive scheduling no task is preempted, otherwise the number of preemptions is finite.

Page 16: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic Scheduling

A location of the task Ti within the schedule:

• completition time Ci

• flow time Fi=Ci–ri,

• lateness Li=Ci–di,

• tardiness Ti=max{Ci–di,0},

• “tardiness flag” Ui=w(Ci>di), i.e. the answer (0/1, yes/no) to the question whether the task is late or not.

Optimization criteria

Page 17: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingOptimization criteria

Most common optimization criteria:

• schedule length Cmax=max{Cj : j=1,...,n},

• sum of completition times Cj = i=1,...,n Ci,

• mean flow time F = (i=1,...,n Fi)/n. M 1

M 2

M 3

T 2

9

T 4 T 1

T 5 T 3

A schedule on three parallel processors, p1,...,p5=6,9,4,1,4.

Cmax = 9,

Cj = 6+9+4+7+8 = 34

In the case there are tasks weights we can consider:

• sum of weighted completition times wjCj = i=1,...,n wiCi,

w1,...,w5=1,2,3,1,1 wjCj = 6+18+12+7+8 = 51

Page 18: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingOptimization criteria

Related to due times:

• maximum lateness Lmax=max{Lj : j=1,...,n},

• maximum tardiness Tmax=max{Tj : j=1,...,n},

• total tardiness Tj = i=1,...,n Ti,

• number of tardy tasks Uj = i=1,...,n Ui,

• weighted criteria may be considered, e.g. total weighted tardiness wjTj = i=1,...,n wiTi. M 1

M 2

M 3

T 2

9

T 4 T 1

T 5 T 3

Lmax = Tmax = 2

Tj = 4, Uj = 2

Li= –1 2 –1 2 0

Ti= 0 2 0 2 0

Task: T1 T2 T3 T4 T5

di= 7 7 5 5 8

Some criteria are pair-wise equivalent

Li = Ci –di, F= (Ci)/n – (ri)/n.

Page 19: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingClassification of deterministic scheduling problems.

| |

Procession environment

Task characteristics

Optimization criterion

is of the form:• P – identical processors

• Q – uniform processors

• R – unrelated processors

• O – open shop

• F – flow shop

• PF – „permutation” flow shop

• J – job shop

Moreover:

• there may be specified the number of processors, e.g. O4,

• in the case of single processors we just put 1,

• we put ‘–’ in the case of processor-free environment.

Page 20: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingClassification of deterministic scheduling problems.

possible values:• pmtn – preemptive tasks,• res – additional resources exist (omitted),• prec – there are precedence constraints,

• rj – arrival times differ per task,

• pj=1 or UET – all processing times equal to 1 unit,

• pij{0,1} or ZUET – all tasks are of unit time or empty (dedicated processors),

• Cjdj – dj denote deadlines,

In the case is empty all tasks characteristics are default: the tasks are non-preemptive, not depended, rj=0, processing times and due dates are arbitrary.

Page 21: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingClassification of deterministic scheduling problems.

possible values:• in–tree, out–tree, chains ... – reflects the precedence constraints (prec).

out–treein–tree

Page 22: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingClassification of deterministic scheduling problems.

Examples.

P3|prec|Cmax – scheduling non-preemptive tasks with precedence constraints on three parallel identical processors to minimize schedule length.

R|pmtn,prec,ri|Ui – scheduling preemptive dependent tasks with arbitrary ready times and arbitrary due dates on parallel unrelated processors to minimize the number of tardy tasks.

1|ri,Cidi|– – decision problem of existence (no optimization criterion) of schedule of independent tasks with arbitrary ready times and deadlines on a single processor, such that no task is tardy.

Page 23: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic Scheduling

• Polynomial-time algorithm – an algorithm that runs in at most w(r) steps for every input of size r bits, for some fixed polynomial w.

• P (Polynomial) – the set of decision problems that can be solved by a polynomial-time deterministic algorithm.

• NP (Non-deterministic Polynomial) – the set of decision problems that can be solved in polynomial time by a non-deterministic algorithm (equivalently: a witness of positive answer can be verified in polynomial time by a deterministic algorithm).

Computational complexity theory notations.

The main hypothesis: P ≠ NP

Page 24: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic Scheduling

• Reduction – a transformation of one problem A to another problem B, i.e. such a transformation of an input of A into some input of B that the answer to A is affirmative iff the answer to B is affirmative too.

• Polynomial reduction – a reduction that can be computed in polynomial time (on deterministic machine).

• NPC (NP-Complete) – the set of NP problems X such that any other NP problem can be polynomialy reduced to X (i.e. constructing a polynomial-time algorithm solving X would bring the P ≠ NP hypothesis down).

Computational complexity theory notations.

Page 25: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic Scheduling

Some NP-complete problems:• Satisfiability Problem – determine if a given Boolean

formula is safisfiable (i.e. there exists such an assignment to the variables that the formula is evaluated to TRUE).

• Partition Problem – for a given sequence of positive integers of even sum S verify if it can be partitioned into two subsequences of sums S/2.

• Clique Problem – for a given graph G and a positive integer k determine if G contains k-vertex complete subgraph.

• Many others...

Computational complexity theory notations.

Page 26: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic Scheduling

• NPH (NP-Hard) – in practice: optimization versions of NPC problems.

• Pseudo-polynomial time algorithm – an algorithm which runs in at most w(r,n1,...,nk) steps for every input of r bits, where ni are the numbers occurring in the input and w is a fixed polynomial.

• Strongly NP-hard, NPH! (strongly NP-complete, NPC!) – NP-hard problems (NP-complete problems) that remain NP-hard (NP-complete) even if all the numbers in the input are bounded by some polynomial of variable r (the size of the input in bits), i.e. believed to be not solvable by any pseudo-polynomial time algorithm.

Computational complexity theory notations.

Page 27: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Example. Pseudo-polynomial algorithm for the Partition Problem (dynamic programming), O(nS) complexity:

Introduction to Deterministic Scheduling

Input: a sequence A=(a1,...,an) of even sum S.boolean T [0..S/2];T [0] := true;for s:= 1 to S/2 T [s]:= false; for each aiA if ( ai≥s ) then T [s]:= (T [s] or T [s-ai]); end;end;return T [S/2];

• Therefore, the Partition Problem is weakly NP-complete.

• The Clique Problem is strongly NP-complete (no numbers in the input!)

T[s] – the answer to ‘does A contains a subsequence of sum s?’.

Page 28: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingReductions of scheduling problems

1

P

Pm

Q

Qm

R

Rm

pi=1

rj

wiFi

F

wiTi

Ti

wiUi

Ui

Lmax

Cmax

prec

chain

out–treein–tree

Page 29: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Introduction to Deterministic SchedulingComputational complexity of scheduling problems

If we restrict the number of processors to 1,2,3,, there are 4536 problems: 416 – polynomial-time solvable, 3817 – NP–hard, 303 – open.

How do we cope with NP–hardness? polynomial-time approximate algorithms with guaranteed approximation ratio, exact pseudo-polynomial time algorithms, exact algorithms, efficient only in the mean-case, heuristics (tabu-search, genetic algorithms, etc.), in the case of small input data – exponential exhaustive search (e.g. branch and bound).

Page 30: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

General problem analysis schema

Optimization problem X

decision version Xd

Xd P? Xd NPC?Construct effective

algorithm solving X

Xd PseudoP? Xd NPC! ?

Construct pseudo-polynomial time algorithm for X

Do approximations satisfy us?Yes

Polynomial-time• approximate algorithms• approximation schemas

No

Restrict problem X• Small data: exhaustive search (branch & bound)

• Heuristics: tabu search, ...

Do not exist

Introduction to Deterministic Task Scheduling

Page 31: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Critical Path Method

–|prec|Cmax model consists of a set of dependent tasks of arbitrary lengths, which do not need processors. Our aim is to construct a schedule of minimum length.

Precedence relation is a quasi order in the set of tasks, i.e. it is• anti-reflective: Ti TiTi

• transitive Ti,Tj,Tk,(TiTj TjTk) TiTk

Page 32: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Critical Path Method

AN (activity on node) network:• nodes correspond to tasks, nodes weights are equal to processing times,

• TiTj there exists a directed path connecting node Ti and node Tj,• transitive arcs are removed

Precedence relation is represented with an acyclic digraph.

AA (activity on arc) network:• arcs correspond to tasks, their length is equal to processing times,• For each node v there exists a path starting at S (source) and terminating at T (sink) passing through v.

•TiTj arc Ti end-node is the starting-node of Tj, or there exists a directed path starting at Ti end-node and terminating at Tj start-node.• to construct the network one may need to add apparent tasks – zero-length tasks.

Page 33: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Critical Path Method

Example. Precedence relation for 19 tasks.

T1,3

T2,8

T3,2

T4,2

T5,4

T6,6

T7,9

T9,1

T10,2

T8,2

T11,1

T12,2

T14,5

T15,9

T13,6

T16,6

T17,2

T18,5

T19,3

AN networkA

SC G

T

B E H J

D F I

T19,3

T13,6T8,2T4,2

T2,8 T10,2 T15,9

T17,2T12,2T7,9

T18,5T14,5T9,1T5,4T1,3

T3,2 T6,6 T11,1 T16,6

AA network

Precedence relation is represented with an acyclic digraph.

Example. Translating AN AA we may need to add (zero-length) apparent tasks. T1

T2

T3

T4

A

B

T0,p0=0

T3T1

T2 T4

ST

Page 34: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Critical Path Method

–|prec|Cmax model consists of a set of dependent tasks of arbitrary lengths, which do not need processors. Our aim is to construct a schedule of minimum length.The idea: for every task Ti we find the earliest possible start time l(Ti), i.e. the length of the longest path terminating at that task.

How to find these start times?AN network Algorithm : 1. find a topological node ordering (the start of any arc precedes its end),2. assign l(Ta)=0 for every task Ta without predecessor,

3. assign l(Ti)=max{l(Tj)+pj: exists an arc (Tj,Ti)} to all other tasks in topological order.AN network algorithm: 1. find a topological node ordering,2. l(S)=0, assign l(v)=max{l(u)+pj: arc Tj connects u and v} to each node v,

Result: l(Tj) is equal to l(v) of the starting node v of Tj. l(T) is the length of an optimal schedule.

Page 35: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Critical Path Method

S:A:B:C:D:E:F:G:H:I:J:T:

0 S:0+3 3 S:0+2 2 S:0+8, A:3+4, B:2+6 8 A:3+2 5 B:2+911 C:8+1 ,D:5+29 C:8+2, E:11+112 E:11+213 F:9+6, G:12+517 G:12+6, H:13+218 I:17+5, J:18+3, G:12+922

A

SC G

T

B E H J

D F I

T19,3

T13,6T8,2T4,2

T2,8 T10,2 T15,9

T17,2T12,2T7,9

T18,5T14,5T9,1T5,4T1,3

T3,2 T6,6 T11,1 T16,6

Topological

ordering

Starting

times

–|prec|Cmax model consists of a set of dependent tasks of arbitrary lengths, which do not need processors. Our aim is to construct a schedule of minimum length.

Example. Construction of a schedule for 19 tasks

Page 36: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Critical Path Method

5 10 15 20

T1 T4

T3

T2

T6

T5

T7

T9 T8

T10

T11

T12

T13

T14

3

T15

3

T16

3

T17

T18

3

T19

S:A:B:C:D:E:F:G:H:I:J:T:

0 3 2 8 5 11 9 12 13 17 18 22

T1,3

T2,8

T3,2

T4,2

T5,4

T6,6

T7,9

T9,1

T10,2

T8,2

T11,1

T12,2

T14,5

T15,9

T13,6

T16,6T17,2

T18,5

T19,3

A

SC G

T

B E H J

D F I

T19,3

T13,6T8,2T4,2

T2,8 T10,2 T15,9

T17,2T12,2T7,9

T18,5T14,5T9,1T5,4T1,3

T3,2 T6,6 T11,1 T16,6

Page 37: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Critical Path Method

• Critical Path Method does not only minimize Cmax, but also optimizes all previously defined criteria

• We can introduce to the model arbitrary release times by adding for each task Tj extra task of length rj, preceding Tj.

Page 38: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Some Discrete Optimization Problems

maximum flow problem. There is given a loop-free multidigraph D(V,E) where each arc is assigned a capacity c:EN. There are two specified nodes – the source s and the sink t. The aim is to find a flow f:EN{0} of maximum value.

What is a flow of value F?• eE f(e)c(e), (flows may not exceed capacities)• vV–{s,t} e terminates at v f(e) – e starts at v f(e) = 0, (the same flows in and flows out for every ‘ordinary’ node)• e terminates at t f(e) – e starts at t f(e) = F, (F units flows out of the network through the sink)• e terminates at s f(e) – e starts at s f(e) = –F, (F units flows into the network through the source)

Page 39: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Some Discrete Optimization Problems

maximum flow problem. There is given a loop-free multidigraph D(V,E) where each arc is assigned a capacity c:EN. There are two specified nodes – the source s and the sink t. The aim is to find a flow f:EN{0} of maximum value.

2

5

31 1

2 3 2

2 11 4

S T

Network, arcs capacity

Page 40: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Some Discrete Optimization Problems

maximum flow problem. There is given a loop-free multidigraph D(V,E) where each arc is assigned a capacity c:EN. There are two specified nodes – the source s and the sink t. The aim is to find a flow f:EN{0} of maximum value.

2/2

5/2

3/21/0 1/0

2/2 3/1 2/2

2/1 1/11/1 4/1

S T

... and maximum flow F=5

Complexity O(|V||E|log(|V|2/|E|)) O(|V|3).

Page 41: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Some Discrete Optimization Problems

Many graph coloring models. Longest (shortest) path problems. Linear programming – polynomial-time algorithm known.

The problem of graph matching. There is a given graph G(V,E) with a weight function w:EN{0}. A matching is a subset AE of pair-wise non-neighboring edges.

• Maximum matching: find a matching of the maximum possible cardinality ((L(G))). The complexity O(|E||V|1/2).• Heaviest (lightest) matching of a given cardinality. For a given k(L(G)) find a matching of cardinality k and maximum (minimum) possible weight sum.• Heaviest matching. Find a matching of maximum possible weigh sum. The complexity O(|V|3) for bipartite graphs and O(|V|4) in general case.

Page 42: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Some Discrete Optimization Problems

Cardinality: 4Weight: 4

1

101

1 1

1

1

1

1

10 1

1 1

11

1

Cardinality: 3Weight: 12

Maximum matching needs not to be the heaviest one and vice-versa.

Page 43: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

M 1

M 2

M 3

5

M 1

M 2

M 3

Z 1

5

M 1

M 2

M 3

Z 1

5

Z 2

Z 2 M 1

M 2

M 3

Z 1

5

Z 2

Z 2

Z 3

Z 3

M 1

M 2

M 3

Z 1

5

Z 2

Z 2

Z 4 Z 3

Z 3

M 1

M 2

M 3

T 1

5

T 2

T 2

T 5 T 4 T 3

T 3

Scheduling on Parallel Processors to Minimize the Schedule Length

Identical processors, independent tasks

Preemptive scheduling P|pmtn|Cmax.McNaughton Algorithm (complexity O(n))1. Derive optimal length Cmax*=max{j=1,...,n pj/m, max j=1,...,n pj},2. Schedule the consecutive tasks on the first machine until Cmax* is reached. Then interrupt the processed task (if it is not completed) and continue processing it on the next machine staring at the moment 0.Example. m=3, n=5, p1,...,p5=4,5,2,1,2.

i=1,...,5 pi=14, max pi=5, Cmax*=max{14/3,5}=5.

X* denotes the optimal value (i.e. the minimum possible value) of the parameter X, e.g. Cmax*, Lmax*.

Page 44: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Identical processors, independent tasks

Non-preemptive scheduling P||Cmax.The problem is NP–hard even in the case of two processors (P2||Cmax).

Proof. Partition Problem: there is given a sequence of positive integers a1,...an, such that S=i=1,...,n ai. Determine if there exists a sub-sequence of sum S/2?PP P2||Cmax reduction: put n tasks of lengths pj=aj (i=1,...,n) and two

processors. Determine if CmaxS/2.

M1

M2

S/2

p i ...

...

There exists an exact pseudo-polynomial dynamic-programming algorithm of complexity O(nCm), for some CCmax*.

Page 45: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Identical processors, independent tasksNon-preemptive scheduling P||Cmax.Polynomial-time approximation algorithms. List Scheduling LS – an algorithm used in numerous problems:• Fix an ordering of the tasks on the list,• Any time a processor gets free (a task processed by that processor has been completed), schedule the first available task from the list on that processor.

In the case of dependent tasks. Task Ti is available if all the tasks Tj it depends of

(i.e. TjTi) are completed.

Page 46: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Identical processors, independent tasks

Example. m=3, n=5, p1,...,p5=2,2,1,1,3. M1

M2

M3

5

List scheduling Optimal scheduling

M 1

M 2

M 3 T 2

3

T 1

T 5

T 3

T 4

T 1

T 2

T 3 T 4

T 5

List Scheduling LS – an algorithm used in numerous problems:• Fix an ordering of the tasks on the list,• Anytime a processor gets free (a task processed by that processor has been completed), schedule the first available task from the list on that processor.

Non-preemptive scheduling P||Cmax.Polynomial-time approximation algorithms.

Page 47: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Identical processors, independent tasks

Approximation ratio: LS is 2–approximate: Cmax(LS)(2–m–1)Cmax*.Proof (includes dependant tasks model P|prec|Cmax). Consider a sequence of tasks T(1),..., T(k) in a LS schedule, such thatT(1) – the last completed task, T(2) – the last completed predecessor of T(1) etc.

Cmax(LS)

T(1) ... ...

T(2)

C *max(pmtn) Cmax*

Cmax(LS) i=1,...,k p(i)+i pi/m == (1–1/m)i=1,...,k p(i)+ i pi/m

(2–1/m)C *max(pmtn)(2–1/m) Cmax*

Non-preemptive scheduling P||Cmax.Polynomial-time approximation algorithms.

List Scheduling LS – an algorithm used in numerous problems:• Fix an ordering of the tasks on the list,• Anytime a processor gets free (a task processed by that processor has been completed), schedule the first available task from the list on that processor.

Page 48: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

LPT (Longest Processing Time) scheduling:• List scheduling, where the tasks are sorted in non-decreasing processing times pi order.Approximation ratio. LS is 4/3–aproximate:Cmax(LPT)(4/3–(3m)–1)Cmax*.

Unrelated processors, not dependent tasksPreemptive scheduling R|pmtn|Cmax

Polynomial time algororihtm – to be discussed later ...

Non-preemptive scheduling R||Cmax

• The problem is NP–hard (generalization of P||Cmax). • Subproblem Q|pi=1|Cmax is solvable in polynomial time. • LPT is used in practice.

Identical processors, independent tasks

Non-preemptive scheduling P||Cmax.Polynomial-time approximation algorithms.

Page 49: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Preemptive scheduling P|pmtn,prec|Cmax.• The problem is NP–hard. • P2|pmtn,prec|Cmax and P|pmtn,forest|Cmax are solvable in O(n2) time.• The following inequality estimating preemptive, non-preemptive and LS schedules holds

C*max C*(LS) (2–m–1)C*max(pmtn)

Proof. The same as in the case of not dependent tasks.

Identical processors, dependent tasks

Page 50: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Non-preemptive scheduling P|prec|Cmax.• Obviously the problem is NP–hard. • Many unit-time processing time cases are known to be solvable in polynomial time:

P|pi=1,in–forest|Cmax and P|pi=1,out–forest|Cmax (Hu algorithm, complexity O(n)), P2|pi=1,prec|Cmax (Coffman–Graham algorithm, complexity O(n2)),

• Even P|pi=1,opositing–forest|Cmax i P2|pi{1,2},prec|Cmax are NP–hard.Hu algorithm: • out–forest in–forest reduction: reverse the precedence relation. Solve the problem and reverse the obtained schedule.• in–forest in–tree: add extra task dependent of all the roots. After obtaining the solution, remove this node from the schedule.• Hu algorithm sketch: list scheduling for dependent tasks + descending distance from the root order.

Identical processors, dependent tasks

Page 51: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Non-preemptive scheduling

Hu algorithm (P|pi=1,in–tree|Cmax): • Level of a task – number of nodes in the path to the root.• The task is available at the moment t if all the tasks dependent of that task have been completed until t.

Compute the levels of the tasks;t:=1;repeat

Find the list Lt of the tasks available at the moment t;Sort Lt in non-increasing levels order; Assign m (or less) first tasks from Lt to the processors;Remove the scheduled tasks from the graph;t:=t+1;

until all the tasks are scheduled;

Identical processors, dependent tasks

Page 52: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Example. Hu algorithm. n=12, m=3.

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T121

2

3

4 - available tasks

Identical processors, dependent tasks

Non-preemptive scheduling

Page 53: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Example. Hu algorithm. n=12, m=3.

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T121

2

3

4

M 1

M 2

M 3

5

Identical processors, dependent tasks

Non-preemptive scheduling

Page 54: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Example. Hu algorithm. n=12, m=3.

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T121

2

3

4

M 1

M 2

M 3

5

T 1

T 3

T 2

Identical processors, dependent tasks

Non-preemptive scheduling

Page 55: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Example. Hu algorithm. n=12, m=3.

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T121

2

3

4

M 1

M 2

M 3

5

T 1

T 6 T 3

T 2

T 4

T 5

Identical processors, dependent tasks

Non-preemptive scheduling

Page 56: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Example. Hu algorithm. n=12, m=3.

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T121

2

3

4

M 1

M 2

M 3

5

T 1

T 6 T 3

T 2

T 4

T 5

T 9

T 7

T 8

Identical processors, dependent tasks

Non-preemptive scheduling

Page 57: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Example. Hu algorithm. n=12, m=3.

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T121

2

3

4

M 1

M 2

M 3

5

Z 1

Z 6 Z 3

Z 2

Z 4

Z 5

Z 9

Z 7

Z 8

Z 10

Z 11

M 1

M 2

M 3

5

T 1

T 6 T 3

T 2

T 4

T 5

T 9

T 7

T 8

T 10

T 11

T 12

Identical processors, dependent tasks

Non-preemptive scheduling

Page 58: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Coffman–Graham algorithm (P2|pi=1,prec|Cmax): 1. label the tasks with integers l from the range [1 ... n], 2. list scheduling, with an ascending labels order.

Phase 1 – tasks labeling;no task has any label or list at the beginning;for i:=1 to n do begin

A:=the set of tasks without label, for which all dependenttasks are labeled;

for each TA do assign the ascending sequence of the labels of tasks dependent of T to list(T);

choose TA with the lexicographic minimum list(T); list(T):=i;

end;

Identical processors, dependent tasks

Non-preemptive scheduling

Page 59: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

Example. Coffman–Graham algorithm, n=17.

T1

T3

T2

T4

T5

T6

T7

T9

T10

T8

T11

T12

T14

T17

T13

T15

T16

( )

( )

( )

( )

1

(1)

(1)

2

(2)

3

(3,2)

4(4,3,2)

5(5)

6

7(7)

8

9

(9)

(9)

(9,6)

10

11(11,8)

12(12,10)

13

14

15(15,13)

16

17

Identical processors, dependent tasks

Non-preemptive scheduling

Page 60: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

T1

T3

T2

T4

T5

T6

T7

T9

T10

T8

T11

T12

T14

T17

T13

T15

T16

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

The ordering in the list: T2, T1, T7, T3, T6, T5, T12, T4, T10, T11, T16, T9, T8, T17, T14, T15, T13.

M 1

M 2 9

T 12 T 7 T 2

T 1

T 11

T 3

T 6 T 9

T 5

T 17

T 4

T 15

T 10 T 16 T 8 T 14 T 13

Identical processors, dependent tasks

Non-preemptive scheduling

Example. Coffman–Graham algorithm, n=17.

Page 61: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Schedule Length

LS heuristic can be applied to P|prec|Cmax. The solution is 2–approximate: Cmax(LS)(2–m–1)Cmax*.

Proof. It has been proved already ...

The order on the list (priority) can be chosen in many different ways. However, some anomalies may occur, i.e. the schedule may lengthen while: increasing the number of processors, decreasing the processing times, releasing some precedence constraints, changing the list order.

Identical processors, dependent tasks

Non-preemptive scheduling

Page 62: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Mean Flow Time

Identical processors, independent tasks

Corollaries.• the processing time of the first task is multiplied by the greatest coefficient; the coefficients of the following tasks are decreasing,• to minimize Cj we should schedule short tasks first (as they are multiplied by the greatest coefficients),• list scheduling with the SPT rule (Shortest Processing Times) leads to an optimal solution on a single processor,• how to assign the tasks to the processors?

Proposal: task Tj scheduled in the k-th position on machine Mi increments the value of Cj by kpj (or kpij in the case of R|).

M T a T c T b

3 2 1

Page 63: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Both preemptive and non-preemptive casesThe problems P||Ci and P|pmtn|Ci can be considered together (preemptions do not improve the criterion)Optimal algorithm O(nlog n):1. Suppose the number of tasks is a multiplicity of m (introduce empty tasks if needed),2. Sort the tasks according to SPT,3. Assign the following m–tuples of tasks to the processors arbitrarily.

Example. m=2, n=5, p1,...,p5=2,5,3,1,3.

SPT: T4 T1 T3 T5 T2

pi= 1 2 3 3 5

M 1

M 2

8

M 1

M 2

8

Z 4 M 1

M 2

8

Z 4

Z 3

Z 1 M 1

M 2

8

T 4

T 3

T 1

T 5

T 2

M 1

M 2

T 4 T 3

Z 1 T 5

T 2 Other possibility:

9 Cj*=21

Z0

0Empty task

Scheduling on Parallel Processors to Minimize the Mean Flow Time

Identical processors, independent tasks

Page 64: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Proof (the case of non-preemptive scheduling).Lemma. Suppose a1,...,an i b1,...,bn are sequences of positive integers. How to permutate them in order to make the dot product a(1)b(1)+a(2)b(2)+...+a(n–1)b(n–1)+a(n)b(n) • the greatest possible?• the smallest possible?

– both should be sorted in ascending order,– sort one ascending, the second descending

Example. The sequences are (3,2,4,6,1) and (5,7,8,1,2).(1,2,3,4,6) and (1,2,5,7,8) 1+4+15+28+48=96(1,2,3,4,6) and (8,7,5,2,1) 8+14+15+8+6=51

Scheduling on Parallel Processorsto Minimize Mean Flow Time

Both preemptive and non-preemptive casesThe problems P||Ci and P|pmtn|Ci can be considered together (preemptions do not improve the criterion)Optimal algorithm O(nlog n):1. Suppose the number of tasks is a multiplicity of m (introduce empty tasks if needed),2. Sort the tasks according to SPT,3. Assign the following m–tuples of tasks to the processors arbitrarily.

Identical processors, independent tasks

Page 65: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Proof (the case of non-preemptive scheduling). Consider an optimal scheduling. One may assume that there are k tasks scheduled on each processors (introducing empty tasks if needed).

M 1

Mm

T(1)

T(m)

T(m+1)

T(2m)

T(km-m+1)

T(km)

Ci = kp(1)+...+kp(m) ++(k–1)kp(m+1) +...+(k–1)p(2m) + +1kp(km–m+1) +...+1p(km)

Reordering the tasks according to the SPT rule does not increase Ci.

Scheduling on Parallel Processors to Minimize the Mean Flow Time

Both preemptive and non-preemptive casesThe problems P||Ci and P|pmtn|Ci can be considered together (preemptions do not improve the criterion)Optimal algorithm O(nlog n):1. Suppose the number of tasks is a multiplicity of m (introduce empty tasks if needed),2. Sort the tasks according to SPT,3. Assign the following m–tuples of tasks to the processors arbitrarily.

Identical processors, independent tasks

Page 66: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Identical processors, independent tasksNon-preemptive schedulingIn the case the weights are introduced even P2||wjCj is NP–hard.Proof (sketch) Similar to P2||Cmax. PP P2||wiCi reduction: take n tasks with pj= wj=aj (j=1,...,n), two processors. There exists a number C(a1,...,an), such that wjCjC(a1,...,an) Cmax

*= i=1,...,n ai/2 (exercise).

Following Smith rule in the case of single processor scheduling (1||wjCj) leads to an optimal solution in O(nlog n) time:• sort the tasks in ascending pj/wj order.Proof. Consider the improvement of the criterion caused by changing two consecutive tasks. wjpj+wi(pi+pj) – wipi – wj(pi+pj) =

= wipj – wjpi 0 pj/wj pi/wi

violating Smith rule increases wiCi .

M T j T i

Scheduling on Parallel Processors to Minimize the Mean Flow Time

Page 67: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Identical processors, independent tasksNon-preemptive tasks

RPT algorithm can be used in order to minimize both Cmax and Ci :1. Use the LPT algorithm.2. Reorder the tasks within each processor due to the SPT rule. Approximation ratio: 1Ci (RPT)/Ci*m (commonly better)

Identical processors, dependent tasks• Even P|prec,pj=1|Ci, P2|prec,pj{1,2}|Ci, P2|chains|Ci and P2|chains,pmtn|Ci are NP–hard.• Polynomial-time algorithms solving P2|prec,pj=1|Ci (Coffman–Graham) and P|out–tree,pj=1|Ci (adaptation of Hu algorithm) are known.• In the case of weighted tasks even single machine scheduling of unit time tasks 1|prec,pj=1|wiCi is NP–hard.

Scheduling on Parallel Processors to Minimize the Mean Flow Time

Page 68: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Unrelated processors, independent tasksO(n3) algorithm solving R||Ci is based on the problem of graph matching.Bipartite weighted graph: Partition V1 corresponding to the tasks T1,...,Tn. Partition V2 – each processor n times: kMi, i=1...m, k=1...n. The edge connecting Tj and kMi is weighted with kpij (it corresponds to scheduling Tj on Mi in the k–th position from the end.

kpij

T1

Tj

Tn

1M1

1Mm

2M1

nMm

nM1

kMi

2

k

n

1

We construct the lightest matching of n edges, which corresponds to optimal scheduling.

Scheduling on Parallel Processors to Minimize the Mean Flow Time

Page 69: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Properties: •Lmax criterion is a generalization of Cmax, the problems that are NP–hard in the case of minimizing Cmax remain NP-hard in the case of Lmax,

• if we have several tasks of different due times we should start with the most urgent one to minimize maximum lateness,• this leads to the EDD rule (Earliest Due Date) – choose tasks in the ordering of ascending due dates dj,• the problem of scheduling on a single processor (1||Lmax) is solved by using the EDD rule.

M T a T b

da db

M T a T b

da db

Page 70: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Identical processors, independent tasksPreemptive scheduling

Single machine: Liu algorithm O(n2), based on the EDD rule, solves 1|ri,pmtn|Lmax:1. Choose the task of the smallest due date among available ones. 2. Every time a task has been completed or a new task has arrived go to 1. (in the latter case we preempt currently processed task).

Arbitrary number of processors (P|ri,pmtn|Lmax). A polynomial-time algorithm is known:We use sub-routine solving the problem with deadlines P|ri,Cidi,pmtn|–. We find the optimal value of Lmax using binary search algorithm.

Scheduling on Parallel Processorsto Minimize the Maximum Lateness

Page 71: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

We transform P|ri,Cidi,pmtn|– to the problem of maximum flow. First we put the values ri and di into the ascending sequence e0<e1<...<ek.

m(e1–e0)

m(ei–ei–1)S T

w1

wi

wk

T1

Tj

Tn

m(ek–ek–1)

p1

pj

pn

e1–e0

ei–ei–1

We construct a network:• The source is connected by k arcs of capacity m(ei–ei–1) to nodes wi, i=1,...,k.• The arcs of capacity pi connect nodes-tasks Ti to the sink, i=1,...,n.• We connect wi and Tj by an arc of capacity ei–ei–1, iff [ei–1,ei][rj,dj].

A schedule exists there exists a flow of value i=1,...,n pi (one can distribute the processors among tasks in the time intervals [ei-1, ei] to complete all the tasks).

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Page 72: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Some NP–hard cases: P2||Lmax, 1|rj|Lmax.

Dependent tasksNon-preemptive scheduling

Polynomial-time solvable cases:• unit processing times P|pj=1,rj|Lmax. • similarly for uniform processors Q|pj=1|Lmax (the problem can be reduced to linear programming),• single processor scheduling 1||Lmax – solvable using the EDD rule (has been discussed already...).

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Page 73: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Single processor scheduling 1|pmtn,prec,rj|Lmax can be solved by a modified Liu algorithm O(n2):1. Determine modified due dates for each task:

dj*=min{dj, mini{di:TjTi}}2. Apply the EDD rule using dj* values, preempting current task in the case a task with smaller modified due-date gets available.3. Repeat 2 until all the tasks are completed.

Dependent tasksPreemptive scheduling

• Some other polynomial-time cases: P|pmtn,in–tree|Lmax, Q2|pmtn,prec,rj|Lmax. • Moreover, some pseudo-polynomial time algorithms are known.

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Page 74: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

• Even P|pj=1,out–tree|Lmax is NP–hard.• A polynomial-time algorithm for P2|prec,pj=1|Lmax is known.• P|pj=1,in–tree|Lmax can be solved by Brucker algorithm O(nlog n):

Dependent tasksNon-preemptive scheduling

next(j) = immediate successor of task Tj.1. Derive modified due dates: droot*=1–droot for the root and dk*=max{1+dnext(k)*,1–dk} for other tasks2. Schedule the tasks in a similar way as in Hu algorithm, choosing every time the tasks of the smallest modified due date instead of the smallest distance from the root.

The list scheduling again with a different ordering of the tasks again.

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Page 75: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Dependent tasksNon-preemptive scheduling

Example. Brucker algorithm, n=12, m=3, due dates in the nodes.

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T127

6 3

1 5 3 1

4 6 4 2 3

-6

-5,-5 -2,-5

0,-4 -4,-4 -2,-1 0,-1

-3,-3-5,-3

-3,0 -1,1 -2,1

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Page 76: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Maximum Lateness

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T127

6 3

1 5 3 1

4 6 4 2 3

-6

-5 -2

0 -4 -1 0

-3 -3

0 1 1

Dependent tasksNon-preemptive schedulingExample. Brucker algorithm, n=12, m=3, due dates in the nodes.

Page 77: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T12

M 1

M 2

M 3

6

Scheduling on Parallel Processors to Minimize the Maximum Lateness

-6

-5 -2

0 -4 -1 0

-3 -3

0 1 1

Dependent tasksNon-preemptive scheduling

Example. Brucker algorithm, n=12, m=3, due dates in the nodes.

Page 78: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T12

M 1

M 2

M 3

6

T 3

T 5

T 4

-6

-5 -2

0 -4 -1 0

-3 -3

0 1 1

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Dependent tasksNon-preemptive scheduling

Example. Brucker algorithm, n=12, m=3, due dates in the nodes.

Page 79: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T12

M 1

M 2

M 3

6

T 3

T 5

T 4

T 6

T 9

T 8

-6

-5 -2

0 -4 -1 0

-3 -3

0 1 1

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Dependent tasksNon-preemptive scheduling

Example. Brucker algorithm, n=12, m=3, due dates in the nodes.

Page 80: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T12

M 1

M 2

M 3

6

T 3

T 5

T 4

T 6

T 9

T 8

T 1

T 11

T 2

-6

-5 -2

0 -4 -1 0

-3 -3

0 1 1

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Dependent tasksNon-preemptive scheduling

Example. Brucker algorithm, n=12, m=3, due dates in the nodes.

Page 81: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T12

M 1

M 2

M 3

6

T 3

T 5

T 4

T 6

T 9

T 8

T 1

T 11

T 2

T 7

-6

-5 -2

0 -4 -1 0

-3 -3

0 1 1

Scheduling on Parallel Processors to Minimize the Maximum Lateness

Dependent tasksNon-preemptive scheduling

Example. Brucker algorithm, n=12, m=3, due dates in the nodes.

Page 82: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T12

M 1

M 2

M 3

6

Z 3

Z 5

Z 4

Z 6

Z 9

Z 8

Z 1

Z 11

Z 2

Z 7 Z 10

-6

-5 -2

0 -4 -1 0

-3 -3

0 1 1

Scheduling on Parallel Processors to Minimize the Maximum Lateness

M 1

M 2

M 3

6

T 3

T 5

T 4

T 6

T 9

T 8

T 1

T 11

T 2

T 7 T 10 T 12

Dependent tasksNon-preemptive scheduling

Example. Brucker algorithm, n=12, m=3, due dates in the nodes.

Page 83: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Parallel Processors to Minimize the Maximum Lateness

T1 T2 T3 T4 T5

T7T6

T10

T9T8

T11

T127

6 3

1 5 3 1

4 6 4 2 3

M 1

M 2

M 3

6

T 3

T 5

T 4

T 6

T 9

T 8

T 1

T 11

T 2

T 7 T 10 T 12

-1

-1 0

Lmax*=1-1 -1 1

-1 -3

-3 -1 -2

Lateness:

Dependent tasksNon-preemptive scheduling

Example. Brucker algorithm, n=12, m=3, due dates in the nodes.

Page 84: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Minimizing the Number of Tardy Tasks on a Single Machine

Independent non-preemptive tasks

Obviously even P2||Ui and P2||Ti are NP-hard. Proof. Similar to P2||Cmax.

Further we consider single processor scheduling only.

Minimizing number of tardy tasks 1||Ui is polynomial-time solvableHodgson algorithm O(nlog n):

Sort the tasks with the EDD rule: T(1), T(2),...,T(n); A:=;for i:=1 to n do begin

A:=A{T(i)};if ZjA pj>d(i) then remove the longest task from A;

end;A – maximum cardinality set of tasks that can be scheduled on time.

Schedule A in the EDD order, then the rest of the tasks in any order;

Page 85: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Minimizing the Number of Tardy Tasks on a Single Machine

Independent non-preemptive tasks

• Weighted tasks scheduling 1||wiUi is NP–hard as a generalization of knapsack problem (the same for 1||wiTi) .• Assuming unit execution times makes these problems easier: 1|pj=1|wiUi i 1|pj=1|wiTi are polynomial time solvable – a natural reduction to the problem of lightest matching in a bipartite graph.

Page 86: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Minimizing the Number of Tardy Tasks on a Single Machine

Dependent and non-preemptive tasks

NP-hardness even with unit processing times for the problems 1|pj=1,prec|Ui and 1|pj=1,prec|Ti.

Proof. Clique problem: for a given graph G (V,E) and a positive integer k determine if G contains a complete subgraph of k vertices.CP 1|pj=1,prec|Ui reduction: We put unit processing time tasks Tv with dv=|VE| for every vertex vV and Te with de=k+k(k–1)/2 for every edge eE. Precedence constraints: Tv Te v is incident to e. Limit L=|E|–k(k–1)/2.

Example. k=3

a

b

cd

Ta

Tb

Tc

Td

T{a,b}

T{a,c}

T{b,c}

T{c,d}

di=8 di=6

L=1

Page 87: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Minimizing the Number of Tardy Tasks on a Single Machine

M 1

k

T v T v T v T e ... T e ...

k+k(k-1)/2

T ... Z

|VE| All the tasks are completed until |VE| in any optimal solution. Thus, whenever UiL, at least k(k–1)/2 tasks Te are completed until k+k(k–1)/2. Therefore the corresponding edges are incident with at least k vertices (for which the tasks Tv precede Te). That is possible only in the case these k vertices form a clique.

In a similar way PK 1|pj=1,prec|Ti reduction.

Dependent and non-preemptive tasks

NP-hardness even with unit processing times for the problems 1|pj=1,prec|Ui and 1|pj=1,prec|Ti.

Proof. Clique problem: for a given graph G (V,E) and a positive integer k determine if G contains a complete subgraph of k vertices.CP 1|pj=1,prec|Ui reduction: We put unit processing time tasks Tv with dv=|VE| for every vertex vV and Te with de=k+k(k–1)/2 for every edge eE. Precedence constraints: Tv Te v is incident to e. Limit L=|E|–k(k–1)/2.

Page 88: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Minimizing the Number of Tardy Tasks on a Single Machine

Parallel processors, minimizing Cmax again

We have constructed a reduction PK 1|pj=1,prec|Ui. In a similar way NP–hardness of P|pj=1,prec|Cmax can be proved.

Proof. Clique Problem: for a given graph G (V,E) and a positive integer k determine if G contains a complete subgraph of k vertices.CP P|pj=1,prec|Cmax reduction: unit time tasks Tv for each vV and Te for each eE. Precedence constraints: Tv Te v is incident to e. Limit L=3. Moreover, 3 'levels' of unit tasks TA1,TA2,... TB1,TB2,... TC1,TC2,... The number of processors m big enough to make Cmax=3 possible:

3 T A

...

k(k-1)/2+ +|V|-k

k

T B

...

|E|- -k(k-1)/2 +

T C

...

If Cmax*=3 then:• All the gray-colored parts are fulfilled with Tv i Te,• In the 1st time unit only Tv are scheduled, in the 3rd time unit only Te are scheduled,• In the 2nd time unit k(k–1)/2 tasks Te are processed and the corresponding edges are incident to k vertices, which form a clique (corresponding tasks are processed in the first time unit).

Page 89: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated Processors Rehersal jobs consist of tasks preassigned to processors (Tij is a taks of Jj that is preassigned to Mi, its processing time is pij). A job is completed when the last its task is completed, some jobs may not need all the processors (empty operations), no two tasks of the same job can be scheduled in parallel, processors are capable to process at most one task at a time.

Models of scheduling on dedicated processors:• flow shop – all jobs have the same processing order through the machines coincident with machines numbering,• open shop – no predefined machine sequence exists for any job,• other, not discussed ...

Page 90: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Flow shopEven 3-processor scheduling (F3||Cmax) is NP–hard. Proof. Partition Problem: for a given sequence of positive integers a1,...an, S=i=1,...,n ai determine if there exists a sub-sequence of sum S/2.PP F3||Cmax reduction: put n jobs with processing times (0,ai,0) i=1,...,n, and a job (S/2,1,S/2). Determine if these jobs can be scheduled with CmaxS+1.

M1

M2

S+1

a i... a j ...

S/2

S/2

1

M3

Permutation Flow Shop (PF): flow shop + each machine processes jobs in the same order (some permutation of the jobs).

Scheduling on Dedicated Processors

Page 91: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsFlow shop

In a classical flow shop the jobs visit the processors in the same order (coincident with processors numbers), however the sequeneces of jobs within processors may differ (which may occur even in an optimal schedule).

Example. m=4, n=2. Processing times of J1: (1,4,4,1) and of J2 :(4,1,1,4). M 1

M 2

M 3

14

J 1

J 1

M 4 J 1

J 2

J 2

J 1

J 2

J 2

M 1

M 2

M 3

14

J 1

J 1

M 4 J 1

J 2

J 2

J 1

J 2

J 2

Permutation schedules ...

M 1

M 2

M 3

12

J 1

J 1

M 4 J 1

J 2

J 2

J 1

J 2

J 2

and non-permutation schedule

Page 92: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsFlow shop

Suppose pij>0, There exists a schedule, such that the sequence of jobs for the first two processors is the same an for the the last two processors is the same. Corollary. An optimum schedule PFm||Cmax is an optimum solution Fm||Cmax

for m3 and pij>0 (only permutation schedules are to be checked – smaller space of solutions to search!).Proof. The sequence of jobs on M1 can be rearranged to be coincident with the sequence on M2.

M1

M2

T1j T1i

T2i T2j

interchange

Page 93: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsFlow shop

Two processors scheduling F2||Cmax (includes the case of preemptive scheduling F2|pmtn|Cmax), Johnson algorithm O(n log n):1. Partition the set of jobs into two subsets N1={Jj: p1j<p2j}, N2={Jj: p1jp2j}, 2. Sort N1 in non-decreasing p1j order and N2 in non-increasing p2j order,3. Schedule all the jobs on both machines in order of the concatenation sequence N1,N2.

M 1

M 2

10

Z 1 Z2

Z 2

20

Example. Johnson algorithm, m=2, n=5.J1 J2 J3 J4 J5

M1 4 1 4 5 3M2 2 3 4 6 2

N2 N1 N2 N1 N2

N1: J2 J4 1 5

N2: J3 J1 J5 4 2 2

M 1

M 2

10

Z 1 Z2

Z 2

20

Z 1 Z4

1 Z4

M 1

M 2

10

Z 1 Z2

Z 2 Z3

20

Z 1 Z4

1 Z4 Z3

M 1

M 2

10

Z 1 Z2 Z 1

Z 2 Z3

20

Z 1 Z4

1 Z4 Z3

Z1 M 1

M 2

10

J 5

Z 1 J2 J 1

J 2 J3

20

Z 1 J4

1 J4 J3

J1

J 5

Page 94: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated Processors

Proof (sketch). Johnson Lemma. Consider a job sequence such that Ji precedes Jj whenever min{p1i,p2j} min{p2i,p1j}. Then the induced permutation schedule is optimal for F2||Cmax.

Consider a pair of jobs Ji, Jj from the sequence obtained in the algorithm:• both belong to N1: p1i=min{p1i,p2j} min{p2i,p1j},• both belong to N2: p2j=min{p1i,p2j} min{p2i,p1j},• Ji belongs to N1 and Jj belongs to N2: p1i p2i i p2j p1j.Preemptive scheduling case: Parts of interrupted tasks can be merged together without Cmax increase.

Flow shopTwo processors scheduling F2||Cmax (includes the case of preemptive scheduling F2|pmtn|Cmax), Johnson algorithm O(n log n):1. Partition the set of jobs into two subsets N1={Jj: p1j<p2j}, N2={Jj: p1jp2j}, 2. Sort N1 in on-decreasing p1j order and N2 in non-increasing p2j order,3. Schedule all the jobs on both machines in order of the concatenation sequence N1,N2.

Page 95: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated Processors

F||Cmax: polynomial-time "graphical" algorithm for n=2 jobs and arbitrary number of machines. Sketch:1. We put intervals of the length p11, p21, ..., pm1 (processing times of J1) on the

OX axis and we put intervals of the length p12, p22, ..., pm2 (J2 processing times)

on the OY axis.2. We create rectangular obstacles – Cartesian products of corresponding intervals (a processor cannot process two tasks at a time).3. We construct the shortest path consisting of segments parallel to one of the axis (single processor is working) or diagonal in the plane (both processors are working), avoiding passing through any obstacles, from (0,0) to (ipi1, ipi2) – the

distance function is defined by ((x1,x2),(y1,y2))=max{|x1–x2|, |y1–y2|}. The length of

the path is equal to the length of the optimum schedule.

• F2||Cj is NP–hard,• F3||Cmax, in the case M2 is dominated by M1 (i,j p1i p2j), or by M3 (i,j p3ip2j) one can use Johnson algorithm for n jobs with processing times (p1i+p2i, p2i+p3i), i=1,...,n.

Flow shop

Page 96: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsFlow shop

Example. Graphical algorithm. m=4, n=2; J1 processing times (1,4,4,1); J2

processing times: (4,1,1,4).

M 1

M 2

M 3

12

J 1

J 1

M 4 J 1

J 2

J 2

J 1

J 2

J 2

M 1

M 2

M 3

14

J 1

J 1

M 4 J 1

J 2

J 2

J 1

J 2

J 2

10 5

10

5

J2

J1

Page 97: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsOpen shop

The three processors case (O3||Cmax) is NP–hard.

The problem O2||Cj is NP–hard.

Proof. PP O3||Cmax reduction: put n tasks with processing times (0,ai,0) i=1,...,n and three tasks of processing times (S/2,1,S/2), (S/2+1,0,0), (0,0,S/2+1). Determine if there exists a schedule with CmaxS+1.

M1

M2

S+1

a i... a j ...

S/2

S/2

1

M3

S/2+1

S/2+1

M 1

M 2

S+1

a i ... a j ...

S/2

S/2

1

M 3

S/2+1

S/2+1 lub

Page 98: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsOpen shop

The case of two processors O2||Cmax (and O2|pmtn|Cmax), Gonzalez–Sahni algorithm O(n):1. Partition the set of tasks into N1={Jj: p1j<p2j}, N2={Jj: p1jp2j}, 2. Let Jr, Jl, be two jobs such that: p1rmaxJjN2 p2j; p2lmaxJjN1 p1j; 3. p1:=ip1i; p2:=ip2i; N1’:=N1\{Jr,Jl}; N2’:=N2\{Jr,Jl};4. Construct two schedules: jobs N1’{Jl}, Zl as the first; N2’{Jr}, Jr as the last (permutation and no–wait schedules) :

M 1

M 2

1 Jl

Jl

N1’

N1’

M 1

M 2 Jr

Jr

N2’

N2’

5. Merge these schedules. if p1–p1lp2–p2r (p1–p1l<p2–p2r) then ‘shift’ the tasks of N1’{Jl} to the right on M2 else ‘shift’ the tasks of N2’{Jr} to the left on M1; M 1

M 2

1 Jl

Jl

N1’

N1’ Zr

Zr

N2’

N2’

Page 99: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsOpen shop

The case of two processors O2||Cmax (and O2|pmtn|Cmax), Gonzalez–Sahni algorithm O(n):

M 1

M 2

1 Jl

Jl

N1’

N1’

M 1

M 2 Jr

Jr

N2’

N2’

5. Merge these schedules. if p1–p1lp2–p2r (p1–p1l<p2–p2r) then ‘shift’ the tasks of N1’{Jl} to the right on M2

else ‘shift’ the tasks of N2’{Jr} to the right on M1; [*] M 1

M 2

1 Jl

Jl

N1’

N1’ Zr

Zr

N2’

N2’

6. Move the task of Jr on M2 ([*] Jl on M1) to the beginning ([*] the end) and then to the right ([*] to the left).

Jl

Jl

N1’

N1’ Jr

Jr

N2’

N2’

Jl

Jl

N1’

N1’ Jr

Jr

N2’

N2’ or

Cmax==max{p1,p2}

Cmax==max{p1,p1r+ p2r}

Page 100: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsOpen shop

Example. Gonzalez–Sahni algorithm, m=2, n=5.J1 J2 J3 J4 J5

M1 4 1 4 5 3M2 2 3 4 6 2

N2 N1 N2 N1 N2

N1: J2, J4 N2: J1, J3, J5

p1rmaxJjN2 p2j=4 p2lmaxJjN1 p1j=5

Jr:=J1 Jl:=J4

N1’: J2

N2’: J3, J5

J2

J 2 J4

J4

J 5

J 1

J3

J3

J1

J 5

J2

J 2 J4

J4

J 5

J 1

J3

J3

J1

J 5 Merge:

M 1

M 2

10 17

J2

J 2 J4

J4

J 5

J 1

J3

J3

J1

J 5

Page 101: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsOpen shop

Zero or unit processing times (O|ZUET|Cmax): polynomial-time algorithm based on edge-coloring of bipartite graphs.

M1

Jj

Jn

Mi

Mm

J1

1. Bipartite G:a) one partition corresponds to the jobs set; the other represents the processors,b) Each non-empty task Oij corresponds to an edge {Jj,Mi}.

2. We edge-color this graph using (G) colors. The colors are interpreted as the time units in which the corresponding tasks are scheduled, (proposal: feasible schedule proper coloring).3. Then, Cmax*= (G) =max{maxij=1,...,npij,maxji=1,...,mpij}. Obviously, no shorter schedule exists.

Page 102: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsOpen shop

Preemptive scheduling (O|pmtn|Cmax): pseudo-polynomial time algorithm similar to the algorithm for O|ZUET|Cmax. We construct a bipartite multigraph G, i.e. each non-empty task Tij corresponds to pij parallel edges Again Cmax* =max{maxij=1,...,npij,maxji=1,...,mpij}.

Example. Preemptive scheduling m=3, n=5, p1=(2,3,0), p2=(1,1,1), p3=(2,2,2), p4=(0,1,3), p5=(1,0,1).

M1 M2 M3

J1 J2 J3 J4 J5

Why ‘pseudo’? The number of the edges may be non-polynomial (=i=1,...,m; j=1,...,n pij), the schedule may contain non-polynomial number of interrupts.

Page 103: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsOpen shop

42 1

4 67

45 6 3 7

5

61 2

35 3 1 2

M1 M2 M3

J1 J2 J3 J4 J5

M 3

M 2

M 1

7

J 2

J 3

J 2

J 3

J 3

J 2

J 3 J 4

J 5

J 5

J 1 J 3

J 1

J 4

Preemptive scheduling (O|pmtn|Cmax): pseudo-polynomial time algorithm similar to the algorithm for O|ZUET|Cmax. We construct a bipartite multigraph G, i.e. each non-empty task Tij corresponds to pij parallel edges Again Cmax* =max{maxij=1,...,npij,maxji=1,...,mpij}.

Example. Preemptive scheduling m=3, n=5, p1=(2,3,0), p2=(1,1,1), p3=(2,2,2), p4=(0,1,3), p5=(1,0,1).

Why ‘pseudo’? The number of the edges may be non-polynomial (=i=1,...,m; j=1,...,n pij), the schedule may contain non-polynomial number of interrupts.

Page 104: Deterministic Scheduling Krzysztof Giaro and Adam Nadolski Gdańsk University of Technology

Scheduling on Dedicated ProcessorsOpen shop

Preemptive scheduling (O|pmtn|Cmax):• Polynomial-time algorithm is known; it is based on fractional edge-coloring of weihted graph (each task Tij corresponds to an edge {Jj,Mi} of weight pij in graph G ),

Minimizing Cmax on parallel processors ... again

Polynomial-time algorithm for R|pmtn|Cmax.R|pmtn|Cmax O|pmtn|Cmax. reduction: Let xij be the part of Tj processed by Mi (in the time tij= pijxij). If we knew optimal values xij, we could use the above algorithm treating these tasks parts as tasks of preemptive jobs (constraints for the schedule are the same!).How to derive these values? We transform the scheduling problem to linear programming:

minimize C such that:i=1,...,m xij=1, j=1,...,n

Cj=1,...,n pijxij, i=1,...,m, Ci=1,...,m pijxij, j=1,...,n.