ADVANCED ALGORITHMS LECTURE 5: DYNAMIC PROGRAMMING

�1

LECTURE 5

ANNOUNCEMENTS

▸ Homework 1 due on Monday Sep 10, 11:59 PM

▸ Instructor office hours Thursday -> Friday (1 - 2 pm)

▸ Comment on notes — JeffE’s material

▸ Solutions to HW0 on course page

▸ Pecking order problem of HW1

�2

vertices birdsO O

could be cycles

O O

LECTURE 5

LAST CLASS

�3

▸ Divide and conquer

▸ Break instance into smaller pieces, solve pieces separately, combine solutions (*)

▸ Merge-sort, multiplying n digit numbers (time << n2)

▸ Median (and k’th smallest element) in O(n) time (no sorting!)

▸ Key idea: approximate median to break array into “nearly equal” pieces

▸ All analyses by writing recursion, solving (tree, plug-and-chug, …)

n58

1Master theorem

Notes onhomepageAkira Bazzitheorem Jeff E's

notes

LECTURE 5

OVERLAP IN RECURSIVE CALLS

�4

▸ Divide & conquer: usually sub-problems don’t interact “much”

▸ What if they do? Can run into wastefulness — solving same sub-problem many times…

▸ Motivation for dynamic programming

LECTURE 5

TODAY’S PLAN

�5

▸ Continue example — subset sum

▸ Knapsack problem

▸ Traveling salesman

LECTURE 5

RECAP: SUBSET SUM

�6

Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.

▸ Trivial algorithm: 2n time — check all subsets

▸ Sometimes better: divide into two halves — check all possibilities for how sum is divided

T(n) = S ⋅ T(n/2) ⟹ T(n) = Slog2 n = nlog2 S

Sn sL s I

Alo All f Aln i

It fl nitlogg2

5 30 i 5 7 4 15 38 221 29

LECTURE 5

TRY 2

�7

▸ See if first element is picked or not

Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.

timeonwhenwehauf.in

Suppose we can write S Afi 1 Afia tafiacorrectnessis

Alo isNOT Alo isused immediate we

usedtowrite fw arepdqkijy.ua.lt

f ESee ifwe canwrite

seeif S Alo can bewritten as

5 using All Aln i using Afi An i

I

LECTURE 5

RUNNING TIME

�8

c n runtime when we have

m Afit's

1 n 2T n l I Thi

Th F fintheomania In Fast

2 2 t t 2 t ln

nEE 2 2 almost

as bad as bruteforce

LECTURE 5

LOOKING MORE CLOSELY — EXAMPLE

�9

A[] = {1, 2, 3, 5, 7, 9, 10, 11}; S = 20TEt is

3,5 20 E3 B b 19

3 5.3 s 20 3,5 18 3 3,51 17N L I I

sit 17 5,7 s 17

1start index _2Sum needed 17

LECTURE 5

AVOIDING MULTIPLE SOLVES ON “SAME” INSTANCE

�10

▸ Store answers!

▸ How much are we storing?

▸ Running time

Howmany differentsub problems

can one have 7 cc 2

if I

any n possibilities possible

of distinct instances is E n sa

Modi thm

In each recursive step firstcheck if we have solved thisinstance before If so read offthe answer

If not we store the answer aftercomputation

Answers to computation can be stored in ann Sti array

start index Sam needed

LECTURE 5

IS THIS POLYNOMIAL?

�11

subtle point

when we say polytime algorithm we mean

polynomial in INPUT SIZE go

01 all an i S S 2

og att flog aft flog.amif ilgRunning time n.SE e udaiistn

Input size an Cfznz.sn andsaySERunning time n I fan n 2

LECTURE 5

“KNAPSACK” / COIN CHANGE

�12

Problem: suppose we are given coins of denominations 1c, 5c, 10c, 20c, 25c, 50c. Make change for $1.90 using the fewest # of coins.

▸ Heuristics?

a

206 201C

50 50d 50k 254,104,5

GreedyAtways use the largest coin that youCAN use

E

LECTURE 5

RECURSIVE ALGORITHM

�13

denominations do d dn

target sum S

Son'tused use do once The dotwice

1eachis a recursive call

s using d idnB and

7 do using it 3 tankandHans 2 S 2do 3

i2tans3n

LECTURE 5

RUNNING TIME

�14

All recursive calls are of the type

S set of denominationsI1

Sum needed suffix of doD

Space needed to store answers 01ns

Time needed Ofn S Edo

LECTURE 5

GENERAL IDEA OF DYNAMIC PROGRAMMING

�15

▸ Write down recursive algorithm

▸ Observe “common sub-problems” in recursion

▸ Capture sub-problems “succinctly”

LECTURE 5

TRAVELING SALESMAN PROBLEM

�16

Problem: given an undirected graph with edge weights, find a path that visits every vertex precisely once and returns to start.

LECTURE 5

TRAVELING SALESMAN PROBLEM

�17

Problem: given an undirected graph with edge weights, find a path that visits every vertex precisely once and returns to start.

▸ Trivial algorithm?

LECTURE 5

TRAVELING SALESMAN PROBLEM

�18

Problem: given an undirected graph with edge weights, find a path that visits every vertex precisely once and returns to start.

▸ Trivial algorithm?

▸ Many heuristics for geometric cases

LECTURE 5

RECURSIVE ALGORITHM

�19

LECTURE 5

RECURSIVE ALGORITHM — CORRECTNESS

�20

LECTURE 5

RECURSIVE ALGORITHM — CORRECTNESS

�21

LECTURE 5

SUB-PROBLEMS, RUNNING TIME

�22

LECTURE 5

SUB-PROBLEMS, RUNNING TIME

�23

LECTURE 5

CAN WE DO BETTER?

�24

▸ Likely not — NP hard problem

▸ We will see that finding “approximately good” solution possible!