byung duk song, jonghoe kim, and james r. morrison*

ICUAS 2014

Towards Real Time Scheduling for Persistent UAV Service: A Rolling Horizon MILP Approach, RHTA and the STAH Heuristic

Byung Duk Song, Jonghoe Kim, and James R. Morrison*

Department of Industrial and Systems Engineering KAIST, South Korea

Thursday, May 29, 2014

ICUAS 2014 – 2

Presentation Overview

• Motivation

• Goal and ideas

• Mathematical formulation

• Solution approaches: STAH & RHTA

• Numerical examples

• Concluding remarks

ICUAS 2014 – 3


• Motivation

• Goal and ideas





ICUAS 2014 – 4

Motivation

• Increasing demand for commercial small size UAVs

• Applications: Patrol, tracking, communication relay, environmental / fire / national boundary monitoring, cartography, disaster relief

• Limitations: Flight time & mission duration

• Long duration, multiple customers and persistent service require- Collection of UAVs, refueling stations, automatic operation system- Methods to orchestrate their operations

ICUAS 2014 – 5

Motivation: Application Example (1)

• Example 1: Tracking a moving ground target

Station 1

Station 2

Station 3

ICUAS 2014 – 6

Motivation: Application Example (2)

• Example 2: UAV border patrol

Station 1

Station 2

ICUAS 2014 – 7


• Motivation

• Goal and ideas





ICUAS 2014 – 8

Goal and Ideas

• Provide uninterrupted UAV service with continuous appearance of mission requests (or unanticipated system state changes)

• Persistent UAV operation via service stations

• Real time service perspective: - Incorporate current UAV information (location, fuel level) - Efficient solving algorithms

• Uninterrupted customer service

■ Goal

■ Ideas

ICUAS 2014 – 9


• Motivation

• Goal and ideas





ICUAS 2014 – 10

Mathematical Formulation: Assumptions

■ Assumptions

1. Moving target’s path and location at specific times are known.

2. Current location and fuel levels are known.

3. Recharge time for a UAV depends on the remaining fuel(battery) amount

4. UAV travel speed is constant

ICUAS 2014 – 11

Mathematical Formulation: Split Job

Start

End

Service station 1

Service station 2

1

2

3

104

5

6 7

89

Job 1

UAV 1UAV 1

UAV 2

UAV 2

▪ Objective moves

- From point (50,250) to (950,350)

- From 13:10 to 13:20

Split jobStart point

End point

Start time

End time

1 50,250 150,250 13:10 13:11

2 150,250 250,250 13:11 13:12

3 250,250 350,250 13:12 13:13

4 350,250 450,250 13:13 13:14

5 450,250 550,250 13:14 13:15

6 550,250 650,250 13:15 13:16

7 650,250 750,250 13:16 13:17

8 750,250 850,250 13:17 13:18

9 850,250 950,250 13:18 13:19

10 950,250 950,350 13:19 13:20

■ Split job: Moving objective path is divided into set of split jobs

ICUAS 2014 – 12

Mathematical Formulation: Notation

■ Notationi, j : Indices for jobs

s : Index for stations

k : Index for UAVs

r : Index of a UAV’s rth flight

NJ: Number of split jobs

NUAV: Number of UAVs in the system

NSTA: Number of recharge stations

NR: Maximum number of flight per UAV during the time horizon

M : Large positive number

(xjs, yj

s) : Start point of split job j

(xje, yj

e) : End point of split job j

Dij: Distance from split job ith finish point to split job jth start point, Dij ≠ Dji

Ei : Start time of split job i

Pi : Processing time or split job i

qk: Maximum traveling time of UAV k

Sok: Initial location(station) of UAV k

TSk: Travel speed of UAV k

H : Required time for fully recharge(refuel) the empty fuel tank (battery)

qk,ini: Initial remaining battery (fuel) of UAV k

ICUAS 2014 – 13

Mathematical Formulation: Notation

■ Notation

ΩJ : = {1, …, NJ}, Set of split jobs

ΩINI : = Set of initial UAV location

ΩSS : = {NJ+1, NJ+3, …, NJ+2∙ NSTA-1}, set of start station

ΩSE : = {NJ+2, NJ+4, …, NJ +2∙ NSTA}, set of end station

ΩA : = (ΩD U ΩSS U ΩSE) = {1,…, NJ+2∙NSTA}, set of all jobs and recharge stations

■ Decision Variables

▪ Xijkr = 1 if UAV k processes split job j or recharges at station j after processing split job i or recharging at station i during the rth flight; 0, otherwise

▪ Cikr is job i’s start time by UAV k during its rth flight or UAV k’s recharge start time at station i; otherwise its value is 0.

▪ qkr is total fuel (battery) consumption amount for UAV k during its rth flight

ICUAS 2014 – 14

Mathematical Formulation

■ Mathematical formulation

1, 1 ( , 1... 1, )J SS J SS

iskr s ikr R SEi i

X X k K r N s

1 ( , )SE J SS

iskrs i

X k K r R

1 ( , )J INI J SE

sjkrs j

X k K r R

, 1 1 ( )ok

J SE

s jkj

X k K

Subject to

1, 1 ( , 1... 1, )skr s kr R SEC C k K r N s

1 ( )A

ijkr Jk K r R i

X j

0 ( , , )A A

ijkr jikr Jj j

X X i k K r R

0 ( , , )J SS

iskr SS INIi

X k K r R s

A A

ij ijkrk K r R i j

D X

Minimize (1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

Network flow &

job assignment

ICUAS 2014 – 15

Mathematical Formulation

/ (1 ) ( , , , )ikr i ij k jkr ijkr J SS J SEC P D TS C M X i j k K r R

( )ikr i Jk K r R

C E i

/ ( , )A A J A

ij k ijkr i ijkr kri j i j

D TS X P X q k K r R

0 ( , , )ikr AC k K r R i

{0,1} ( , , , )ijkr A AX k K r R i j

(10)

(11)

(12)

(13)

(14)

(15)

, ( , 1)kr k iniq q k K r

( , , 1)kr kq q k K r R r

0 ( , )krq k K r R

(16)

(17)

(18)

(19)

( , , )J SE

ijkr ikr J SSj

M X C i k K r R

1 ,( ( )) / / (1 ) ( , , , 2)ikr kr k k ini k ij k jkr ijkr SS J SEC H q q q q D TS C M X i j k K r

1 / / (1 ) ( , , , 2)ikr kr k ij k jkr ijkr SS J SEC H q q D TS C M X i j k K r

(20)

Start time constraints

Fuel con-straints

Decision vari-ables

ICUAS 2014 – 16

Formulation Demonstration (CPLEX)

■ Problem description : Problem map

0

50

100

150

200

250

300

0 100 200 300 400 500 600 700

customer 1

customer 2

station

initial location

Customer 1 Customer 2 Station

▲ UAVU1S1

S2

S3

U2

U3

U4U5

U6

ICUAS 2014 – 17


■ Problem description : Split job & station & UAV information

Station x y

S1 394 126

S2 170 79

S3 72 229

UAV x ySpeed

(meter / min)qk,ini

(min)

1 560 130 240 3

2 280 180 240 8

3 170 79 240 8

4 300 100 240 6

5 394 126 240 6

6 90 200 240 8

Customer Split job

Start point End point Start time

x y x y

1

1 596 167 532 161 5

2 532 161 483 129 6

3 438 129 432 94 7

4 432 94 372 91 8

5 372 91 315 87 9

6 315 87 262 74 10

7 262 74 218 99 11

8 218 99 171 134 12

9 171 134 142 186 13

10 142 186 102 225 14

11 102 225 54 251 15

12 54 251 6 266 16

213 458 64 479 105 8

14 479 105 514 63 9

15 514 63 536 15 10

ICUAS 2014 – 18


■ Result of example 1

UAV Scheduling qini qk1 qk2

1 Initial → 1,2,3,4,5 → Station 1 6 5.58 0

2 Initial → 13, 14, 15 → Station 1 8 4.63 0

3 Initial (Station 2) 8 0 0

4 Initial → 6,7,8,9,10,11,12 → Station 3 10 7.4 0


6 Initial → Station 3 8 0.14 0

- Objective value : 662.570- CPU time: 5.42 seconds

ICUAS 2014 – 19


■ Result of example 2 : Change qk,ini


1 Initial location → 1,2 → Station 1 3 2.59 0

2 Initial location → Station 1 → 3,4,5,6,7,8,9,10,11,12→ Station 3 8 0.52 10.7

3 Initial location (Station 2) → Station 2 8 0 0

4 Initial location → 13,14,15 → Station 1 6 4.4 0

5 Initial location (Station 1) → Station 1 6 0 0

6 Initial location → Station 3 8 0.14 0

- Objective value : 809.067- CPU time: 9.17 seconds

ICUAS 2014 – 20


■ These examples shows us

- Real time perspectives: Current location and fuel level of UAVs

- Access to the persistent service

- Sharing of multiple stations

ICUAS 2014 – 21


■ Computational limit of CPLEX

System parameters CPLEX

NJ NSTA NUAVCPUtime

Obj.value

15 3 6 5.42 662.570

30 3 6 N/A N/A

45 3 6 N/A N/A

45 5 10 N/A N/A

60 3 6 N/A N/A

60 5 10 N/A N/A

Needs for alternative solution approaches to achieve real time perspective

ICUAS 2014 – 22


• Motivation

• Goal and ideas





ICUAS 2014 – 23

Solution Approach: Solution Properties

■ Analysis on solutions

1) Strict condition: Feasibility issue - Fuel - Split job start time

2) Flexible condition: Solution quality issue - Total moving distance - The number of split jobs which UAV can serve in that flight - Trade off between direct & indirect flight

ICUAS 2014 – 24


■ Analysis on solutions

3) Sequential assignment of split jobs


1 Initial → 1,2,3,4,5 → Station 1 6 5.58 0

2 Initial → 13, 14, 15 → Station 1 8 4.63 0


4 Initial → 6,7,8,9,10,11,12 → Station 3 10 7.4 0


6 Initial → Station 3 8 0.14 0

ICUAS 2014 – 25


■ Trade off between direct and indirect flight

Service station 1

1

2

3

104

5

6 7

89

UAV 1

< Direct flight >

- Without replenishment- Short moving distance- Serve less number of split jobs

ICUAS 2014 – 26


■ Trade off between direct and indirect flight

< Indirect flight >

- Replenishment on the station- Long moving distance - Serve more number of split jobs

Service station 1

1

2

3

104

5

6 7

89

UAV 1

ICUAS 2014 – 27

Solution Approach: STAH

■ Based on the analysis of solutions

1) Strict condition

2) Flexible condition

Assign UAVs which satisfy strict conditions (Ajk)

Contribution value of direct and indirect flight

VD(k) = {αw( # of split jobs) –(1-α)(total moving distance) }∙Ajk

VI(k) = {αw( # of split jobs) –(1-α)(total moving distance) }∙Ajk

Select the flight which provide highest value

1, if UAV satisfy fuel and start time constraints at split job

0, Otherwise jk

k jA

From customer 1 (earliest customer), split jobs are assigned sequentially

ICUAS 2014 – 28

Solution Approach: RHTA

■ Receding horizon task assignment algorithm (RHTA)

– Classical heuristic (petal algorithm) for vehicle routing problem

– Iterative IP model

– Generate good quality feasible solution in short time

– Classical RHTA was modified: • Eliminate the resource selection components• Modify the equation of replenishment time for a UAV

ICUAS 2014 – 29


■ Petal : Sequence of split jobs to be served by a UAV

• Step 1: Enumerate all feasible petals for each UAV k– Generate petal– Check feasibility of petal

– Evaluate of petal value

)()(/)),(()( 11 wEkTSwkcLDkat

)()(/),()()( 111 iiiii wEkTSwwDwPwE

cc ni

iinc

ni

iii krFwPsWDwwDwkcLD

1

1

111 )()(),(),()),((

Fuel limitation

Time window

ICUAS 2014 – 30


• STEP 2: Solve a single IP to select petal

• STEP 3: Assign split job of selected petal to UAVs

• STEP 4: Send exhausted UAVs to a service station - if there are remaining jobs on the list, go to STEP 1.

• STEP 5: Send any UAVs not located at a station to a station

A job can be processed by at most one UAV

At least P job should be selected

Minimize total travel distance

UAV

1 1

N

1 1

s.t. 1 ( )

kp

kp

Nk K

kip kpk p

Nk

kip kpi W k p

A Y i W

A Y P

1 1

Min kpUAV

NN

kp kpk p

S X

ICUAS 2014 – 31


• Motivation

• Goal and ideas





ICUAS 2014 – 32

Results for Various Problem Sizes

■ Results

System parameters CPLEX STAH RHTA

NJ NSTA NUAVCPUtime

Obj.value

CPUtime

Obj.value

GapCPUtime

Obj.value

Gap

15 3 6 5.42 662.570 0.004 665.261 0.406 % 0.203 664.857 0.345 %

30 3 6 N/A N/A 0.008 1010.24 - 0.327 1274.991 -

45 3 6 N/A N/A 0.015 1314.34 - 0.458 1901.816 -

45 5 10 N/A N/A 0.015 1346.96 - 0.608 1656.667 -

60 3 6 N/A N/A 0.016 1642.34 - 0.578 2677.189 -

60 5 10 N/A N/A 0.031 1565.90 - 0.765 1997.992 -

- STAH and RHTA obtain very near optimal values.

- CPLEX issues an out of memory error.

- STAH and RHTA derive feasible solution in a reasonably short time.

ICUAS 2014 – 33

Real Time Operation with New Service Request Arrival

Customer Split job

Start point End point Start time

x y x y

1

1’ 262 74 218 99 11

2’ 218 99 171 134 12

3’ 171 134 142 186 13

4’ 142 186 102 225 14

5’ 102 225 54 251 15

6’ 54 251 6 266 16

3

7’ 413 210 430 184 13

8’ 430 184 454 162 14

9’ 454 162 482 137 15

10’ 482 137 451 10 16

11’ 451 10 442 82 17

12’ 442 82 428 65 18

UAV x y qk qk,ini TSk

1 394 126 12 12 240

2 262 74 12 7.629 240

3 170 79 12 12 240

4 536 15 12 2.325 240

5 394 126 12 12 240

6 72 229 12 12 240

■ Remaining jobs and UAV information at T=11.

■ During the UAV service of example 1, new service request arrived at T=10.

ICUAS 2014 – 34


Initial plan

UAV Scheduling qk,iniObj.value

CPUTime (sec)

1 Initial location → 1, 2 → Station 1 3

809.067 9.17

2 Initial location → Station 1 → 3,4,5,6,7,8,9,10,11,12 → Station 3 8

3 Initial location (Station 2) 8

4 Initial location → 13, 14, 15 → Station 1 6

5 Initial location (Station 1) 6

6 Initial location → Station 3 8

New plan from time 11

UAV Scheduling qk,iniObj.value

CPUTime (sec)

1 (394,126) (Station 1) 12

411.306 4.67

2 (262,74) → 1’,2’,3’,4’,5’,6’ → Station 3 7.629

3 (170,79) (Station 2) 12

4 (536,15) → Station 1 2.325

5 (394,126) (Station 1) → 7’,8’,9’,10’,11’,12’ → Station 1 12

6 (72,229) (Station 3) 12

ICUAS 2014 – 35

0 100 200 300 400 500 600 7000

50

100

150

200

250

300


Station 1

Station 2

Station 3

T =

4

Customer 1

Customer 2

5678910

Customer 3

111213141516171819

ICUAS 2014 – 36


• Motivation

• Goal and ideas





ICUAS 2014 – 37

Concluding Remarks

• Fleets of UAVs conducting missions over a field of operations can achieve persistence - Supported by shared service stations distributed over the field - Real time operation to handle new arrival of service request

• MILP scheduling model - Allows a mobile robot to perform persistent service by sharing base for energy resupply - Maximize service quality by serving 100% service requests by orchestration of components - Demonstration of real time perspectives

• Develop a efficient heuristics to overcome computational limit of MILP formulation - STAH - RHTA

• Application & numerical examples

ICUAS 2014

Back-UP Materials

ICUAS 2014 – 39

Solution approach: STAH

■ STAH procedure

• STEP 1: Arrange customers in the non-decreasing order of (first) split job start time– Let P= {1,2,…,p} be the set of customers and Pt be the set of split jobs in

customer index t– Element of Pt also arranged according to the non-decreasing order of split job

start time. Set P=1.

• STEP 2: From customer P, split jobs are assigned to the UAVs.– Calculate VD(k) and VI(k)

– Among VD(k) and VI(k) for every UAV, the biggest function value and corresponding UAV schedule is selected

• STEP 3: Update parameters– Current location, fuel level, available time of UAVs– Remaining split job information for customer P is updated

ICUAS 2014 – 40


■ STAH procedure

• STEP 4: Repeat step 2 to 3 until every split job is assigned to UAV in customer P.

• STEP 5: Set P=P+1, and repeat step 2,3 and 4. This step is repeated until P=|P|+1.


ICUAS 2014 – 41


■ Pseudo code (STAH)

ICUAS 2014 – 42

Feasibility

Y

Y

Y

Y

N

Solution approach: RHTA

■ Petal : sequence of split jobs to be served by a UAV

• Step 1: Enumerate all feasible petals for each UAV k– Generate petal (Limitation of maximum size of petal)– Check feasibility of petal

– Evaluate value of petal

Petal size

Instance of petal

1 {2}

1 {3}

2 {2,3}

3 {6,7,8}

4 {1,2,3,4}

)()(/)),(()( 11 wEkTSwkcLDkat

)()(/),()()( 111 iiiii wEkTSwwDwPwE

cc ni

iinc

ni

iii krFwPsWDwwDwkcLD

1

1

111 )()(),(),()),((

Fuel limitation

Time window

Travel dis-tance

6

10

(6+3)

(10+4+5)

infeasible

ICUAS 2014 – 43


• STEP 2: Solve a single IP for all UAVs to minimize the travel distance – Select petals to be used

subject to

1 1

1 ( )kpNk K

kip kpk p

A Y i W

A job can be processed by at most one UAV

At least P job should be selectedUAVN

1 1

kpNk

kip kpi W k p

A Y P

Minimize total travel distance𝑀𝑖𝑛∑𝑘=1

𝑁𝑈𝐴𝑉

∑𝑝=1

𝑁 𝑘𝑝

𝑆𝑘𝑝 𝑋𝑘𝑝

ICUAS 2014 – 44


• STEP 3: Assign split job to UAV– Assign UAV to a first split job of selected petal– Update system state (UAV location, fuel, available time and job information)– Remove assigned job from remaining job list

• STEP 4: Find exhausted UAVs– Send UAVs that do not have any feasible petals to a service station– Update system state – Return to STEP 1 if remaining job list is not empty


ICUAS 2014 – 45


■ Pseudo code (RHTA)

byung duk song, jonghoe kim, and james r. morrison*

Documents

persistent uav service

split job jth

split job jxje

time of split j

uav border patrol station

persistent service

number of split jobsnuav

end point of split job