1A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Frank NIELSEN

nielsenlixpolytechniquefr

A Concise andPracticalIntroduction to ProgrammingAlgorithms in Java

Chapter 9 Combinatorial optimization algorithms

2A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

A few algorithms and paradigms

- I Exhaustive search

- II Greedy algorithm (set cover problems)

- III Dynamic programming (knapsack)

Agenda

+Merging two lists

3A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Linked lists

public class Listint containerList next a reference to a cell of a list Constructor List(head tail) Build a cell so that the reference to the next cell is the tailList(int element List tail)

thiscontainer=element thisnext=tail

next=null

4A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Merging ordered linked lists Two linked lists u and v of increasing integers

Build a new linked list in increasing order

using only cells of u and v (no cell creation new)

U | 3--gt6--gt8--gtnullV | 2--gt4--gt5--gt7--gt9--gtnullMerge(UV) | 2--gt3--gt4--gt5--gt6--gt7--gt8--gt9--gtnull

For example

5A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Linked listsclass Listint containerList next

Constructor List(head tail)List(int element List tail)

thiscontainer=element thisnext=tail

List insert(int el) insert element at the head of the listreturn new List(elthis)

void Display()List u=thiswhile(u=null)

Systemoutprint(ucontainer+--gt)u=unext

Systemoutprintln(null)

U | 3--gt6--gt8--gtnullV | 2--gt4--gt5--gt7--gt9--gtnull

6A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class MergeList Merge two ordered listsstatic List mergeRec(List u List v)if (u==null) return vif (v==null) return uif (ucontainer lt vcontainer)

Recycle cells no newunext=mergeRec(unextv)return u

else Recycle cellsno newvnext=mergeRec(uvnext)return v

public static void main(String [] args)List u=new List(8null)u=uinsert(6)u=uinsert(3)uDisplay()List v=new List(9null)v=vinsert(7)v=vinsert(5)v=vinsert(4)v=vinsert(2)vDisplay()List w=mergeRec(uv)wDisplay()

7A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static List sortRec(List u)int il=ulength() lrList l1 l2 split psplit references to cells

if (llt=1) return u

elsel1=u

psplit=split=ui=0lr=l2

while (iltlr) i++psplit=split split=splitnext

l2=split terminates with a nullpsplitnext=null

return mergeRec( sortRec(l1) sortRec(l2) )

Sort in O(n log n) time Split list in two halves Recursively apply sorting Merge ordered lists

8A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)

List u=new List(3null)u=uinsert(2)u=uinsert(9)u=uinsert(6)u=uinsert(1)u=uinsert(15)u=uinsert(17)u=uinsert(23)u=uinsert(21)u=uinsert(19)u=uinsert(20)

uDisplay()

List sortu=sortRec(u)Systemoutprintln(Sorted linked list)sortuDisplay()

20--gt19--gt21--gt23--gt17--gt15--gt1--gt6--gt9--gt2--gt3--gtnullSorted linked list1--gt2--gt3--gt6--gt9--gt15--gt17--gt19--gt20--gt21--gt23--gtnull

9A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search (Brute force search) The Eight Queens puzzle

92 distinct solutions 12 non-naive distinct solutions (rotationsymmetry) Good exercise for designing algorithms Generalize to n-queens

Max Bezzel (1848 chess player)Find safe positions of 8 queens

on a 8x8 chessboard

2A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

A few algorithms and paradigms

- I Exhaustive search

- II Greedy algorithm (set cover problems)

- III Dynamic programming (knapsack)

Agenda

+Merging two lists

3A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Linked lists

public class Listint containerList next a reference to a cell of a list Constructor List(head tail) Build a cell so that the reference to the next cell is the tailList(int element List tail)

thiscontainer=element thisnext=tail

next=null

4A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Merging ordered linked lists Two linked lists u and v of increasing integers

Build a new linked list in increasing order

using only cells of u and v (no cell creation new)

U | 3--gt6--gt8--gtnullV | 2--gt4--gt5--gt7--gt9--gtnullMerge(UV) | 2--gt3--gt4--gt5--gt6--gt7--gt8--gt9--gtnull

For example

5A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Linked listsclass Listint containerList next

Constructor List(head tail)List(int element List tail)

thiscontainer=element thisnext=tail

List insert(int el) insert element at the head of the listreturn new List(elthis)

void Display()List u=thiswhile(u=null)

Systemoutprint(ucontainer+--gt)u=unext

Systemoutprintln(null)

U | 3--gt6--gt8--gtnullV | 2--gt4--gt5--gt7--gt9--gtnull

6A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class MergeList Merge two ordered listsstatic List mergeRec(List u List v)if (u==null) return vif (v==null) return uif (ucontainer lt vcontainer)

Recycle cells no newunext=mergeRec(unextv)return u

else Recycle cellsno newvnext=mergeRec(uvnext)return v

public static void main(String [] args)List u=new List(8null)u=uinsert(6)u=uinsert(3)uDisplay()List v=new List(9null)v=vinsert(7)v=vinsert(5)v=vinsert(4)v=vinsert(2)vDisplay()List w=mergeRec(uv)wDisplay()

7A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static List sortRec(List u)int il=ulength() lrList l1 l2 split psplit references to cells

if (llt=1) return u

elsel1=u

psplit=split=ui=0lr=l2

while (iltlr) i++psplit=split split=splitnext

l2=split terminates with a nullpsplitnext=null

return mergeRec( sortRec(l1) sortRec(l2) )

Sort in O(n log n) time Split list in two halves Recursively apply sorting Merge ordered lists

8A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)

List u=new List(3null)u=uinsert(2)u=uinsert(9)u=uinsert(6)u=uinsert(1)u=uinsert(15)u=uinsert(17)u=uinsert(23)u=uinsert(21)u=uinsert(19)u=uinsert(20)

uDisplay()

List sortu=sortRec(u)Systemoutprintln(Sorted linked list)sortuDisplay()

20--gt19--gt21--gt23--gt17--gt15--gt1--gt6--gt9--gt2--gt3--gtnullSorted linked list1--gt2--gt3--gt6--gt9--gt15--gt17--gt19--gt20--gt21--gt23--gtnull

9A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search (Brute force search) The Eight Queens puzzle

92 distinct solutions 12 non-naive distinct solutions (rotationsymmetry) Good exercise for designing algorithms Generalize to n-queens

Max Bezzel (1848 chess player)Find safe positions of 8 queens

on a 8x8 chessboard

3A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Linked lists

public class Listint containerList next a reference to a cell of a list Constructor List(head tail) Build a cell so that the reference to the next cell is the tailList(int element List tail)

thiscontainer=element thisnext=tail

next=null

4A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Merging ordered linked lists Two linked lists u and v of increasing integers

Build a new linked list in increasing order

using only cells of u and v (no cell creation new)

U | 3--gt6--gt8--gtnullV | 2--gt4--gt5--gt7--gt9--gtnullMerge(UV) | 2--gt3--gt4--gt5--gt6--gt7--gt8--gt9--gtnull

For example

5A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Linked listsclass Listint containerList next

Constructor List(head tail)List(int element List tail)

thiscontainer=element thisnext=tail

List insert(int el) insert element at the head of the listreturn new List(elthis)

void Display()List u=thiswhile(u=null)

Systemoutprint(ucontainer+--gt)u=unext

Systemoutprintln(null)

U | 3--gt6--gt8--gtnullV | 2--gt4--gt5--gt7--gt9--gtnull

6A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class MergeList Merge two ordered listsstatic List mergeRec(List u List v)if (u==null) return vif (v==null) return uif (ucontainer lt vcontainer)

Recycle cells no newunext=mergeRec(unextv)return u

else Recycle cellsno newvnext=mergeRec(uvnext)return v

public static void main(String [] args)List u=new List(8null)u=uinsert(6)u=uinsert(3)uDisplay()List v=new List(9null)v=vinsert(7)v=vinsert(5)v=vinsert(4)v=vinsert(2)vDisplay()List w=mergeRec(uv)wDisplay()

7A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static List sortRec(List u)int il=ulength() lrList l1 l2 split psplit references to cells

if (llt=1) return u

elsel1=u

psplit=split=ui=0lr=l2

while (iltlr) i++psplit=split split=splitnext

l2=split terminates with a nullpsplitnext=null

return mergeRec( sortRec(l1) sortRec(l2) )

Sort in O(n log n) time Split list in two halves Recursively apply sorting Merge ordered lists

8A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)

List u=new List(3null)u=uinsert(2)u=uinsert(9)u=uinsert(6)u=uinsert(1)u=uinsert(15)u=uinsert(17)u=uinsert(23)u=uinsert(21)u=uinsert(19)u=uinsert(20)

uDisplay()

List sortu=sortRec(u)Systemoutprintln(Sorted linked list)sortuDisplay()

20--gt19--gt21--gt23--gt17--gt15--gt1--gt6--gt9--gt2--gt3--gtnullSorted linked list1--gt2--gt3--gt6--gt9--gt15--gt17--gt19--gt20--gt21--gt23--gtnull

9A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search (Brute force search) The Eight Queens puzzle

92 distinct solutions 12 non-naive distinct solutions (rotationsymmetry) Good exercise for designing algorithms Generalize to n-queens

Max Bezzel (1848 chess player)Find safe positions of 8 queens

on a 8x8 chessboard

4A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Merging ordered linked lists Two linked lists u and v of increasing integers

Build a new linked list in increasing order

using only cells of u and v (no cell creation new)

U | 3--gt6--gt8--gtnullV | 2--gt4--gt5--gt7--gt9--gtnullMerge(UV) | 2--gt3--gt4--gt5--gt6--gt7--gt8--gt9--gtnull

For example

5A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Linked listsclass Listint containerList next

Constructor List(head tail)List(int element List tail)

thiscontainer=element thisnext=tail

List insert(int el) insert element at the head of the listreturn new List(elthis)

void Display()List u=thiswhile(u=null)

Systemoutprint(ucontainer+--gt)u=unext

Systemoutprintln(null)

U | 3--gt6--gt8--gtnullV | 2--gt4--gt5--gt7--gt9--gtnull

6A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class MergeList Merge two ordered listsstatic List mergeRec(List u List v)if (u==null) return vif (v==null) return uif (ucontainer lt vcontainer)

Recycle cells no newunext=mergeRec(unextv)return u

else Recycle cellsno newvnext=mergeRec(uvnext)return v

public static void main(String [] args)List u=new List(8null)u=uinsert(6)u=uinsert(3)uDisplay()List v=new List(9null)v=vinsert(7)v=vinsert(5)v=vinsert(4)v=vinsert(2)vDisplay()List w=mergeRec(uv)wDisplay()

7A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static List sortRec(List u)int il=ulength() lrList l1 l2 split psplit references to cells

if (llt=1) return u

elsel1=u

psplit=split=ui=0lr=l2

while (iltlr) i++psplit=split split=splitnext

l2=split terminates with a nullpsplitnext=null

return mergeRec( sortRec(l1) sortRec(l2) )

Sort in O(n log n) time Split list in two halves Recursively apply sorting Merge ordered lists

8A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)

List u=new List(3null)u=uinsert(2)u=uinsert(9)u=uinsert(6)u=uinsert(1)u=uinsert(15)u=uinsert(17)u=uinsert(23)u=uinsert(21)u=uinsert(19)u=uinsert(20)

uDisplay()

List sortu=sortRec(u)Systemoutprintln(Sorted linked list)sortuDisplay()

20--gt19--gt21--gt23--gt17--gt15--gt1--gt6--gt9--gt2--gt3--gtnullSorted linked list1--gt2--gt3--gt6--gt9--gt15--gt17--gt19--gt20--gt21--gt23--gtnull

9A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search (Brute force search) The Eight Queens puzzle

92 distinct solutions 12 non-naive distinct solutions (rotationsymmetry) Good exercise for designing algorithms Generalize to n-queens

Max Bezzel (1848 chess player)Find safe positions of 8 queens

on a 8x8 chessboard

5A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Linked listsclass Listint containerList next

Constructor List(head tail)List(int element List tail)

thiscontainer=element thisnext=tail

List insert(int el) insert element at the head of the listreturn new List(elthis)

void Display()List u=thiswhile(u=null)

Systemoutprint(ucontainer+--gt)u=unext

Systemoutprintln(null)

U | 3--gt6--gt8--gtnullV | 2--gt4--gt5--gt7--gt9--gtnull

6A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class MergeList Merge two ordered listsstatic List mergeRec(List u List v)if (u==null) return vif (v==null) return uif (ucontainer lt vcontainer)

Recycle cells no newunext=mergeRec(unextv)return u

else Recycle cellsno newvnext=mergeRec(uvnext)return v

public static void main(String [] args)List u=new List(8null)u=uinsert(6)u=uinsert(3)uDisplay()List v=new List(9null)v=vinsert(7)v=vinsert(5)v=vinsert(4)v=vinsert(2)vDisplay()List w=mergeRec(uv)wDisplay()

7A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static List sortRec(List u)int il=ulength() lrList l1 l2 split psplit references to cells

if (llt=1) return u

elsel1=u

psplit=split=ui=0lr=l2

while (iltlr) i++psplit=split split=splitnext

l2=split terminates with a nullpsplitnext=null

return mergeRec( sortRec(l1) sortRec(l2) )

Sort in O(n log n) time Split list in two halves Recursively apply sorting Merge ordered lists

8A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)

List u=new List(3null)u=uinsert(2)u=uinsert(9)u=uinsert(6)u=uinsert(1)u=uinsert(15)u=uinsert(17)u=uinsert(23)u=uinsert(21)u=uinsert(19)u=uinsert(20)

uDisplay()

List sortu=sortRec(u)Systemoutprintln(Sorted linked list)sortuDisplay()

20--gt19--gt21--gt23--gt17--gt15--gt1--gt6--gt9--gt2--gt3--gtnullSorted linked list1--gt2--gt3--gt6--gt9--gt15--gt17--gt19--gt20--gt21--gt23--gtnull

9A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search (Brute force search) The Eight Queens puzzle

92 distinct solutions 12 non-naive distinct solutions (rotationsymmetry) Good exercise for designing algorithms Generalize to n-queens

Max Bezzel (1848 chess player)Find safe positions of 8 queens

on a 8x8 chessboard

6A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class MergeList Merge two ordered listsstatic List mergeRec(List u List v)if (u==null) return vif (v==null) return uif (ucontainer lt vcontainer)

Recycle cells no newunext=mergeRec(unextv)return u

else Recycle cellsno newvnext=mergeRec(uvnext)return v

public static void main(String [] args)List u=new List(8null)u=uinsert(6)u=uinsert(3)uDisplay()List v=new List(9null)v=vinsert(7)v=vinsert(5)v=vinsert(4)v=vinsert(2)vDisplay()List w=mergeRec(uv)wDisplay()

7A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static List sortRec(List u)int il=ulength() lrList l1 l2 split psplit references to cells

if (llt=1) return u

elsel1=u

psplit=split=ui=0lr=l2

while (iltlr) i++psplit=split split=splitnext

l2=split terminates with a nullpsplitnext=null

return mergeRec( sortRec(l1) sortRec(l2) )

Sort in O(n log n) time Split list in two halves Recursively apply sorting Merge ordered lists

8A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)

List u=new List(3null)u=uinsert(2)u=uinsert(9)u=uinsert(6)u=uinsert(1)u=uinsert(15)u=uinsert(17)u=uinsert(23)u=uinsert(21)u=uinsert(19)u=uinsert(20)

uDisplay()

List sortu=sortRec(u)Systemoutprintln(Sorted linked list)sortuDisplay()

20--gt19--gt21--gt23--gt17--gt15--gt1--gt6--gt9--gt2--gt3--gtnullSorted linked list1--gt2--gt3--gt6--gt9--gt15--gt17--gt19--gt20--gt21--gt23--gtnull

9A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search (Brute force search) The Eight Queens puzzle

92 distinct solutions 12 non-naive distinct solutions (rotationsymmetry) Good exercise for designing algorithms Generalize to n-queens

Max Bezzel (1848 chess player)Find safe positions of 8 queens

on a 8x8 chessboard

7A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static List sortRec(List u)int il=ulength() lrList l1 l2 split psplit references to cells

if (llt=1) return u

elsel1=u

psplit=split=ui=0lr=l2

while (iltlr) i++psplit=split split=splitnext

l2=split terminates with a nullpsplitnext=null

return mergeRec( sortRec(l1) sortRec(l2) )

Sort in O(n log n) time Split list in two halves Recursively apply sorting Merge ordered lists

8A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)

List u=new List(3null)u=uinsert(2)u=uinsert(9)u=uinsert(6)u=uinsert(1)u=uinsert(15)u=uinsert(17)u=uinsert(23)u=uinsert(21)u=uinsert(19)u=uinsert(20)

uDisplay()

List sortu=sortRec(u)Systemoutprintln(Sorted linked list)sortuDisplay()

20--gt19--gt21--gt23--gt17--gt15--gt1--gt6--gt9--gt2--gt3--gtnullSorted linked list1--gt2--gt3--gt6--gt9--gt15--gt17--gt19--gt20--gt21--gt23--gtnull

9A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search (Brute force search) The Eight Queens puzzle

92 distinct solutions 12 non-naive distinct solutions (rotationsymmetry) Good exercise for designing algorithms Generalize to n-queens

Max Bezzel (1848 chess player)Find safe positions of 8 queens

on a 8x8 chessboard

8A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)

List u=new List(3null)u=uinsert(2)u=uinsert(9)u=uinsert(6)u=uinsert(1)u=uinsert(15)u=uinsert(17)u=uinsert(23)u=uinsert(21)u=uinsert(19)u=uinsert(20)

uDisplay()

List sortu=sortRec(u)Systemoutprintln(Sorted linked list)sortuDisplay()

20--gt19--gt21--gt23--gt17--gt15--gt1--gt6--gt9--gt2--gt3--gtnullSorted linked list1--gt2--gt3--gt6--gt9--gt15--gt17--gt19--gt20--gt21--gt23--gtnull

9A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search (Brute force search) The Eight Queens puzzle

92 distinct solutions 12 non-naive distinct solutions (rotationsymmetry) Good exercise for designing algorithms Generalize to n-queens

Max Bezzel (1848 chess player)Find safe positions of 8 queens

on a 8x8 chessboard

9A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search (Brute force search) The Eight Queens puzzle

92 distinct solutions 12 non-naive distinct solutions (rotationsymmetry) Good exercise for designing algorithms Generalize to n-queens

Max Bezzel (1848 chess player)Find safe positions of 8 queens

on a 8x8 chessboard

10A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search amp BacktrackingBrute force (naive) algorithmCheck all 64x63xx578 = 283274583552 solutions

Easy to check that a configuration is not safe(check horizontalverticaldiagonal lines)

rarr Two queens cannot be on the same line

Therefore incrementally place queen i (07) on the i-th row on the first free column

If there is no more free columns left then backtrackbacktrack

11A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search amp BacktrackingIncrementally place queen i (07) on the first free columnIf there is no more free columns left then backtrackbacktrack

Consider the previous queen position and increment itscolumn position etc etc etc

until we find a solution (=reach a successful location for queen indexed 7)

queen 1D Array that specifies the column positionQueen i is located at position (iqueen[i])

(with i ranging from 0 to 7)

search Static function that returns aall solution(s)

12A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Are queen (i1j1) and queen (i2j2) safe static boolean WrongPos(int i1 int j1 int i2 int j2) same row same col same diagreturn (i1==i2 ||

j1==j2 || Mathabs(i1-i2) == Mathabs(j1-j2))

Place safely queen i at column jstatic boolean FreeMove(int i int j)boolean result=true

for(int k=0kltik++)result=resultampampWrongPos(ijkqueen[k])

return result

Check for the queensplaced so far

on the chessboard

Static functions to check for collisions

13A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean search(int row)boolean result=false

if (row==n) Terminal case DisplayChessboard() nbsol++ else Exhaustive searchint j=0

while(result ampamp jltn)

if (FreeMove(rowj))

queen[row]=jresult=search(row+1) RECURSIONBACKTRACK

j++ explore all columns

return result

Increment the numberof found solutions

(static class variable)

14A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] arguments)nbsol=0search(0) Call exhaustive search procedureSystemoutprintln(Total number of solutions+nbsol)

static final int n=8static int [] queen=new int[n]static int nbsol

static void DisplayChessboard()int ijSystemoutprintln()for(i=0iltni++)

for(j=0jltnj++)

if (queen[i]=j) Systemoutprint(0)else Systemoutprint(1)

Systemoutprintln()

15A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization Set Cover Problem (SCP)

httpwwwcssunysbedu~algorithfilesset-covershtml

bull Given a graphndash A finite set X = 1 hellip nndash A collection of subsets of S S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m such that Uj in T Sj= Xndash Minimize |T|

Elements of X

Chosensubsets

All elements are coveredRedundant covering

16A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Applications of set cover problems

Choose base stations for quality of service (cell phone network etc)

Discretize terrain using a regular grid (set X)

Each position of antenna -gt subset of covered grid areas (S)

Minimize the number of chosen antennas (X)

Wave propagation patterns of a single antenna

Mountains City

17A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

bull Given a graphndash A finite set X = 1 hellip n (=regular grid elements)ndash A collection of subsets of S (=antenna patterns)

S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m (=subset of antennas)

such that Uj in T Sj= Xndash Minimize |T|

Covered once

Covered twice

Covered thrice

18A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

GREEDY-SET-COVER(XS)1 M X all elements must be coveredlarr2 C Oslash chosen subsetslarr3 while M ne Oslash do4 select an Q Є S that maximizes |Q ח M|5 M M ndash Qlarr6 C C U Qlarr7 return C

A greedy algorithm (algorithme glouton)

Set Cover solution

Input

Output

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

11A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search amp BacktrackingIncrementally place queen i (07) on the first free columnIf there is no more free columns left then backtrackbacktrack

Consider the previous queen position and increment itscolumn position etc etc etc

until we find a solution (=reach a successful location for queen indexed 7)

queen 1D Array that specifies the column positionQueen i is located at position (iqueen[i])

(with i ranging from 0 to 7)

search Static function that returns aall solution(s)

12A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Are queen (i1j1) and queen (i2j2) safe static boolean WrongPos(int i1 int j1 int i2 int j2) same row same col same diagreturn (i1==i2 ||

j1==j2 || Mathabs(i1-i2) == Mathabs(j1-j2))

Place safely queen i at column jstatic boolean FreeMove(int i int j)boolean result=true

for(int k=0kltik++)result=resultampampWrongPos(ijkqueen[k])

return result

Check for the queensplaced so far

on the chessboard

Static functions to check for collisions

13A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean search(int row)boolean result=false

if (row==n) Terminal case DisplayChessboard() nbsol++ else Exhaustive searchint j=0

while(result ampamp jltn)

if (FreeMove(rowj))

queen[row]=jresult=search(row+1) RECURSIONBACKTRACK

j++ explore all columns

return result

Increment the numberof found solutions

(static class variable)

14A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] arguments)nbsol=0search(0) Call exhaustive search procedureSystemoutprintln(Total number of solutions+nbsol)

static final int n=8static int [] queen=new int[n]static int nbsol

static void DisplayChessboard()int ijSystemoutprintln()for(i=0iltni++)

for(j=0jltnj++)

if (queen[i]=j) Systemoutprint(0)else Systemoutprint(1)

Systemoutprintln()

15A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization Set Cover Problem (SCP)

httpwwwcssunysbedu~algorithfilesset-covershtml

bull Given a graphndash A finite set X = 1 hellip nndash A collection of subsets of S S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m such that Uj in T Sj= Xndash Minimize |T|

Elements of X

Chosensubsets

All elements are coveredRedundant covering

16A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Applications of set cover problems

Choose base stations for quality of service (cell phone network etc)

Discretize terrain using a regular grid (set X)

Each position of antenna -gt subset of covered grid areas (S)

Minimize the number of chosen antennas (X)

Wave propagation patterns of a single antenna

Mountains City

17A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

bull Given a graphndash A finite set X = 1 hellip n (=regular grid elements)ndash A collection of subsets of S (=antenna patterns)

S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m (=subset of antennas)

such that Uj in T Sj= Xndash Minimize |T|

Covered once

Covered twice

Covered thrice

18A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

GREEDY-SET-COVER(XS)1 M X all elements must be coveredlarr2 C Oslash chosen subsetslarr3 while M ne Oslash do4 select an Q Є S that maximizes |Q ח M|5 M M ndash Qlarr6 C C U Qlarr7 return C

A greedy algorithm (algorithme glouton)

Set Cover solution

Input

Output

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

12A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Are queen (i1j1) and queen (i2j2) safe static boolean WrongPos(int i1 int j1 int i2 int j2) same row same col same diagreturn (i1==i2 ||

j1==j2 || Mathabs(i1-i2) == Mathabs(j1-j2))

Place safely queen i at column jstatic boolean FreeMove(int i int j)boolean result=true

for(int k=0kltik++)result=resultampampWrongPos(ijkqueen[k])

return result

Check for the queensplaced so far

on the chessboard

Static functions to check for collisions

13A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean search(int row)boolean result=false

if (row==n) Terminal case DisplayChessboard() nbsol++ else Exhaustive searchint j=0

while(result ampamp jltn)

if (FreeMove(rowj))

queen[row]=jresult=search(row+1) RECURSIONBACKTRACK

j++ explore all columns

return result

Increment the numberof found solutions

(static class variable)

14A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] arguments)nbsol=0search(0) Call exhaustive search procedureSystemoutprintln(Total number of solutions+nbsol)

static final int n=8static int [] queen=new int[n]static int nbsol

static void DisplayChessboard()int ijSystemoutprintln()for(i=0iltni++)

for(j=0jltnj++)

if (queen[i]=j) Systemoutprint(0)else Systemoutprint(1)

Systemoutprintln()

15A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization Set Cover Problem (SCP)

httpwwwcssunysbedu~algorithfilesset-covershtml

bull Given a graphndash A finite set X = 1 hellip nndash A collection of subsets of S S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m such that Uj in T Sj= Xndash Minimize |T|

Elements of X

Chosensubsets

All elements are coveredRedundant covering

16A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Applications of set cover problems

Choose base stations for quality of service (cell phone network etc)

Discretize terrain using a regular grid (set X)

Each position of antenna -gt subset of covered grid areas (S)

Minimize the number of chosen antennas (X)

Wave propagation patterns of a single antenna

Mountains City

17A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

bull Given a graphndash A finite set X = 1 hellip n (=regular grid elements)ndash A collection of subsets of S (=antenna patterns)

S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m (=subset of antennas)

such that Uj in T Sj= Xndash Minimize |T|

Covered once

Covered twice

Covered thrice

18A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

GREEDY-SET-COVER(XS)1 M X all elements must be coveredlarr2 C Oslash chosen subsetslarr3 while M ne Oslash do4 select an Q Є S that maximizes |Q ח M|5 M M ndash Qlarr6 C C U Qlarr7 return C

A greedy algorithm (algorithme glouton)

Set Cover solution

Input

Output

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

13A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean search(int row)boolean result=false

if (row==n) Terminal case DisplayChessboard() nbsol++ else Exhaustive searchint j=0

while(result ampamp jltn)

if (FreeMove(rowj))

queen[row]=jresult=search(row+1) RECURSIONBACKTRACK

j++ explore all columns

return result

Increment the numberof found solutions

(static class variable)

14A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] arguments)nbsol=0search(0) Call exhaustive search procedureSystemoutprintln(Total number of solutions+nbsol)

static final int n=8static int [] queen=new int[n]static int nbsol

static void DisplayChessboard()int ijSystemoutprintln()for(i=0iltni++)

for(j=0jltnj++)

if (queen[i]=j) Systemoutprint(0)else Systemoutprint(1)

Systemoutprintln()

15A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization Set Cover Problem (SCP)

httpwwwcssunysbedu~algorithfilesset-covershtml

bull Given a graphndash A finite set X = 1 hellip nndash A collection of subsets of S S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m such that Uj in T Sj= Xndash Minimize |T|

Elements of X

Chosensubsets

All elements are coveredRedundant covering

16A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Applications of set cover problems

Choose base stations for quality of service (cell phone network etc)

Discretize terrain using a regular grid (set X)

Each position of antenna -gt subset of covered grid areas (S)

Minimize the number of chosen antennas (X)

Wave propagation patterns of a single antenna

Mountains City

17A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

bull Given a graphndash A finite set X = 1 hellip n (=regular grid elements)ndash A collection of subsets of S (=antenna patterns)

S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m (=subset of antennas)

such that Uj in T Sj= Xndash Minimize |T|

Covered once

Covered twice

Covered thrice

18A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

GREEDY-SET-COVER(XS)1 M X all elements must be coveredlarr2 C Oslash chosen subsetslarr3 while M ne Oslash do4 select an Q Є S that maximizes |Q ח M|5 M M ndash Qlarr6 C C U Qlarr7 return C

A greedy algorithm (algorithme glouton)

Set Cover solution

Input

Output

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

14A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] arguments)nbsol=0search(0) Call exhaustive search procedureSystemoutprintln(Total number of solutions+nbsol)

static final int n=8static int [] queen=new int[n]static int nbsol

static void DisplayChessboard()int ijSystemoutprintln()for(i=0iltni++)

for(j=0jltnj++)

if (queen[i]=j) Systemoutprint(0)else Systemoutprint(1)

Systemoutprintln()

15A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization Set Cover Problem (SCP)

httpwwwcssunysbedu~algorithfilesset-covershtml

bull Given a graphndash A finite set X = 1 hellip nndash A collection of subsets of S S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m such that Uj in T Sj= Xndash Minimize |T|

Elements of X

Chosensubsets

All elements are coveredRedundant covering

16A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Applications of set cover problems

Choose base stations for quality of service (cell phone network etc)

Discretize terrain using a regular grid (set X)

Each position of antenna -gt subset of covered grid areas (S)

Minimize the number of chosen antennas (X)

Wave propagation patterns of a single antenna

Mountains City

17A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

bull Given a graphndash A finite set X = 1 hellip n (=regular grid elements)ndash A collection of subsets of S (=antenna patterns)

S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m (=subset of antennas)

such that Uj in T Sj= Xndash Minimize |T|

Covered once

Covered twice

Covered thrice

18A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

GREEDY-SET-COVER(XS)1 M X all elements must be coveredlarr2 C Oslash chosen subsetslarr3 while M ne Oslash do4 select an Q Є S that maximizes |Q ח M|5 M M ndash Qlarr6 C C U Qlarr7 return C

A greedy algorithm (algorithme glouton)

Set Cover solution

Input

Output

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

15A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization Set Cover Problem (SCP)

httpwwwcssunysbedu~algorithfilesset-covershtml

bull Given a graphndash A finite set X = 1 hellip nndash A collection of subsets of S S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m such that Uj in T Sj= Xndash Minimize |T|

Elements of X

Chosensubsets

All elements are coveredRedundant covering

16A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Applications of set cover problems

Choose base stations for quality of service (cell phone network etc)

Discretize terrain using a regular grid (set X)

Each position of antenna -gt subset of covered grid areas (S)

Minimize the number of chosen antennas (X)

Wave propagation patterns of a single antenna

Mountains City

17A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

bull Given a graphndash A finite set X = 1 hellip n (=regular grid elements)ndash A collection of subsets of S (=antenna patterns)

S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m (=subset of antennas)

such that Uj in T Sj= Xndash Minimize |T|

Covered once

Covered twice

Covered thrice

18A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

GREEDY-SET-COVER(XS)1 M X all elements must be coveredlarr2 C Oslash chosen subsetslarr3 while M ne Oslash do4 select an Q Є S that maximizes |Q ח M|5 M M ndash Qlarr6 C C U Qlarr7 return C

A greedy algorithm (algorithme glouton)

Set Cover solution

Input

Output

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

16A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Applications of set cover problems

Choose base stations for quality of service (cell phone network etc)

Discretize terrain using a regular grid (set X)

Each position of antenna -gt subset of covered grid areas (S)

Minimize the number of chosen antennas (X)

Wave propagation patterns of a single antenna

Mountains City

17A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

bull Given a graphndash A finite set X = 1 hellip n (=regular grid elements)ndash A collection of subsets of S (=antenna patterns)

S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m (=subset of antennas)

such that Uj in T Sj= Xndash Minimize |T|

Covered once

Covered twice

Covered thrice

18A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

GREEDY-SET-COVER(XS)1 M X all elements must be coveredlarr2 C Oslash chosen subsetslarr3 while M ne Oslash do4 select an Q Є S that maximizes |Q ח M|5 M M ndash Qlarr6 C C U Qlarr7 return C

A greedy algorithm (algorithme glouton)

Set Cover solution

Input

Output

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

17A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

bull Given a graphndash A finite set X = 1 hellip n (=regular grid elements)ndash A collection of subsets of S (=antenna patterns)

S1 S2 hellip Sm

bull ProblemProblemndash Find a subset T of 1 hellip m (=subset of antennas)

such that Uj in T Sj= Xndash Minimize |T|

Covered once

Covered twice

Covered thrice

18A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

GREEDY-SET-COVER(XS)1 M X all elements must be coveredlarr2 C Oslash chosen subsetslarr3 while M ne Oslash do4 select an Q Є S that maximizes |Q ח M|5 M M ndash Qlarr6 C C U Qlarr7 return C

A greedy algorithm (algorithme glouton)

Set Cover solution

Input

Output

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

18A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

GREEDY-SET-COVER(XS)1 M X all elements must be coveredlarr2 C Oslash chosen subsetslarr3 while M ne Oslash do4 select an Q Є S that maximizes |Q ח M|5 M M ndash Qlarr6 C C U Qlarr7 return C

A greedy algorithm (algorithme glouton)

Set Cover solution

Input

Output

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

19A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

ElementsX = 1 hellip 6

SubsetsS1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1

3

6 5

4

2

Visualizing a laquo range set raquo

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

20A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Data-structure for the set cover problem

Incidence matrix boolean matrix

X = 1 hellip 6

S1 = 124 S2 = 345 S3 = 136 S4 = 235 S5 = 456 S6 = 13

1 2 3 4 5 61 1 0 1 0 0 S1

0 0 1 1 1 0 S2

1 0 1 0 0 1 S3

0 1 1 0 1 0 S4

0 0 0 1 1 1 S5

1 0 1 0 0 0 S6

N=6 ELEMENTS

M=6 SUBSETS

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

21A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

class SetCoverint nbelementsint nbsubsetsboolean [][] incidenceMatrix

ConstructorSetCover(int nn int mm)thisnbelements=nn thisnbsubsets=mm

incidenceMatrix=new boolean[nbsubsets][nbelements]

for(int i=0iltnbsubsetsi++)for(int j=0jltnbelementsj++)

incidenceMatrix[i][j]=false

void SetSubsets(int [] [] array) Set incidence matrixfor(int j=0jltarraylengthj++)

for(int i=0iltarray[j]lengthi++)incidenceMatrix[j][array[j][i]]=true

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

22A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

void Display()for(int i=0iltnbsubsetsi++)

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) Systemoutprint(1)

else Systemoutprint(0)Systemoutprintln()

public static void main(String [] args)int [][] subsets=013234 02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)

Systemoutprintln(Set cover problem)setcoverDisplay()

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

23A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static boolean [] GreedySCP(SetCover problem)boolean [] result=new boolean[problemnbsubsets]int cover=0 int select

for(int i=0iltproblemnbsubsetsi++) initially no subsetsresult[i]=false

while(cover=problemnbelements) Choose largest not-yet covered subsetselect=problemLargestSubset()result[select]=true

Update covered matrixcover+=problemCover(select) Update incidence matrixproblemUpdate(select)Systemoutprintln(Selected +select+ Number of covered

elements=+cover)problemDisplay()

return result

Greedy algorithm

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

24A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Number of covered element by subset iint Cover(int i)int nbEl=0

for(int j=0jltnbelementsj++)if (incidenceMatrix[i][j]) ++nbEl

return nbEl

Report the current largest subsetint LargestSubset()int i nbel max select

max=-1select=-1

for(i=0iltnbsubsetsi++)nbel=Cover(i)if (nbelgtmax) max=nbel select=i

return select

Methods of class SetCover

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

25A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Methods of class SetCover Update the incidence matrix

void Update(int sel)int ij

for(i=0iltnbsubsetsi++)

if (i=sel) use sel below so dont modify itfor(j=0jltnbelementsj++)

if (incidenceMatrix[sel][j]) incidenceMatrix[i][j]=false

Remove the chosen subset as wellfor(j=0jltnbelementsj++)

incidenceMatrix[sel][j]=false

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

26A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String [] args)int [] [] subsets =013234

02512434502

SetCover setcover=new SetCover(66)setcoverSetSubsets(subsets)Systemoutprintln(Set cover problem)setcoverDisplay()

boolean [] solution=GreedySCP(setcover)Systemoutprint(Solution)for(int i=0iltsetcovernbsubsetsi++)if (solution[i]) Systemoutprint( +i)Systemoutprintln()

Set cover problem110100001110101001011010000111101000Selected 0 Number of covered elements=3000000001010001001001010000011001000Selected 1 Number of covered elements=5000000000000000001000000000001000000Selected 2 Number of covered elements=6000000000000000000000000000000000000Solution 0 1 2

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

27A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization bounds for greedy algorithm

Lower boundApproximation factor can be as big as Omega(log(n))

Difficult to approximate cannot beat (1-eps)Opt unless P=NP

Upper boundApproximation factor is at most H(n)lt=log(n)

2 sets is optimal solutionbut greedy chooses 3 sets here

CoptTlt= Cgreedy lt= ApproximationFactor x Copt

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

28A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

int [] [] subsets=012345678910111213071289345610111213

SetCover setcover=new SetCover(145)Set cover problem1111111000000000000001111111100000010000000110000011000000011110001111Selected 4 Number of covered elements=81110000000000000000001110000100000010000000110000011000000000000000000Selected 3 Number of covered elements=121000000000000000000001000000100000010000000000000000000000000000000000Selected 2 Number of covered elements=140000000000000000000000000000000000000000000000000000000000000000000000Solution 2 3 4

Etc

Easy to build generic exampleswhere greedy does not behave well

with O(log n) approximation ratio

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

29A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack problem (sac a dos)(First version)

Given A set of n Objects O1 On with

corresponding weights W1 Wn

And a bag (knapsack) of capacity W

Find

All the ways of choosing objects to fully fill the bag

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

30A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Filling the knapsack Need to enumerate all possibilities

n objects =gt 2^n choices (2 4 8 16 32 64 )

How to program thisn is a variable

(cannot fix the number of nest loops)

Need to enumerate all combinations

= Exhaustive search

n=42^4=16

1 1 1 11 1 1 01 1 0 11 1 0 01 0 1 11 0 1 01 0 0 11 0 0 00 1 1 10 1 1 00 1 0 10 1 0 00 0 1 10 0 1 00 0 0 10 0 0 0

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

31A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Enumerating A recursive approachstatic void Display(boolean [] tab)for(int i=0ilttablengthi++)

if (tab[i]) Systemoutprint(1 )elseSystemoutprint(0 )

Systemoutprintln()

static void Enumerate(boolean [] selection int pos)if (pos==selectionlength-1)

Display(selection)elsepos++selection[pos]=trueEnumerate(selectionpos)selection[pos]=falseEnumerate(selectionpos)

public static void main(String[] args)int n=4int iboolean [] select=new boolean[n]for(i=0iltni++)

select[i]=falseEnumerate(select-1)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

32A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Fully filling the knapsack

static void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++ keep track of total number of calls

if ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++ total number of solutions

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

final static int n=10 10 objectsstatic int [] weight=23579114132327

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

33A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

final static int n=10 10 objectsstatic int [] weight=23579114132327

public static void main(String [] args)int totalweight=51numbersol=0numbercall=0

Systemoutprintln(Knapsack)boolean [] chosen=new boolean[n]

SolveKnapSack(chosen totalweight 0 0)Systemoutprintln(Total number of solutions+numbersol)Systemoutprintln( calls=+numbercall)

Knapsack2+3+5+7+11+23+0=512+5+7+9+11+4+13+0=512+5+4+13+27+0=512+7+11+4+27+0=512+9+4+13+23+0=512+9+13+27+0=513+5+7+9+4+23+0=513+5+7+9+27+0=513+5+7+13+23+0=513+5+9+11+23+0=517+4+13+27+0=519+11+4+27+0=5111+4+13+23+0=5111+13+27+0=51Total number of solutions14 calls=2029

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

34A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Exhaustive search Branch amp boundstatic void SolveKnapSack(boolean [] chosen int goal int i int total)numbercall++

if (totalgtgoal) return cutif ((igt=chosenlength)ampamp(total=goal)) return

if (total==goal) Display(chosen goal)

numbersol++

elsechosen[i]=true add item firstSolveKnapSack(chosengoali+1total+weight[i])chosen[i]=false and then remove itSolveKnapSack(chosengoali+1total)

Stop recursion if we alreadyexceed the weight amount

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

35A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Fundamental optimization problem

Given a bag capacity (15 kg) maximize the utility (price) of selected objects

(NP-hard problem)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

36A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack optimization problemGiven weights

utility

Optimizesuch that

(Maybe there exists several solutions)

Maximum weight (capacity of bag)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

37A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Knapsack Example

= 12

weightutility

8 objects

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

38A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)

Dynamic programming computes a table from which a solution can be retrieved

Requires a relational equation to deduce solutions progressively

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

39A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming (Knapsack)Let u(ij) be the maximum utility by

taking objects in 1 i with total weight lt=j

If i=1 then u(ij)=0 for jltp1 and u(ij)=A1 for jgt=P1

If igt1u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

Take object Oi gain Ai but leave room Pi

Do not take object Oi

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

40A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

u(ij)=max( u(i-1j) u(i-1j-Pi)+Ai )

weightutility Pmax= 12

Optimization result Maximum utility given Pmax

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

41A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Reading back Solution

Choose (4 11)

Choose (14 11-3=8)

Choose (27=14+138-4=4)

Choose (33=27+64-2=2)

Choose (38=33+5 2-2=0)

From the table chosen solution O8 O7 O5 O4 O1Choose (Utility=0 Pmax=12)

24=24 do not choose

5=5 do not choose

5=5 do not choose

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

42A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static int nbObjects=8static int [] weight=23524631static int [] utility=581461317104static int weightmax=12static int [] [] array

static void SolveDP()int ijarray=new int[nbObjects][weightmax+1]

initialize the first rowfor(j=0jlt=weightmaxj++)

if (jltweight[0]) array[0][j]=0 else array[0][j]=utility[0]

for all other rowsfor(i=1iltnbObjectsi++)

for(j=0jlt=weightmaxj++)

if (j-weight[i]lt0) array[i][j]=array[i-1][j] else

array[i][j]=max( array[i-1][j]array[i-1][j-weight[i]]+utility[i])

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

43A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

static void InterpretArray()int iuwu=0w=weightmax

for(i=nbObjects-1igt=1i--)

if (array[i][w]=array[i-1][w])Systemoutprint((i+1)+ )w=w-weight[i]u=u+utility[i]

if (array[0][w]=0)Systemoutprintln(1)w=w-weight[0]u=u+utility[0]

Systemoutprintln(Cross check+u+ remaining weight +w)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

44A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

public static void main(String[] args)Systemoutprintln(Solving knapsack using the dynamic

programming paradigm)SolveDP()Display()Systemoutprintln(Reading solution)InterpretArray()

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

45A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Dynamic programming binocular stereo matching

Benchmarkhttpvisionmiddleburyedu~scharstereowebresultsphp

DynamicProgramming

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

46A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Optimization A brief summary Exhaustive search recursion but O(2^n) complexity

Can be improved by backtracking (cuts)

Greedy algorithm Polynomial O(n^3) but yields an approximation

Dynamic programming yields an exact solution but requires O(weightxobjects) time(weights should not be too bigs)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

47A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Last but not least Java appletsApplets are special java programs

that can run into your favorite Internet browser

You need to(1) write a web page with ltAPPLETgt ltAPPLETgt tags(2) write and compile the Java applet code (javac)

Advantages of applets are(1) to be accessible worldwide(2) to provide graphics

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

48A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

import javaawtimport javaappletclass Point2Ddouble xyPoint2D()thisx=3000Mathrandom()thisy=3000Mathrandom()public class AppletINF311 extends Applet final static int n=100static Point2D [] set

public void init() int iset=new Point2D[n]for(i=0iltni++)

set[i]=new Point2D()public void paint(Graphics g) int i

for(i=0iltni++)int xi yixi=(int)set[i]x yi=(int)set[i]ygdrawRect(xi yi11) gdrawString(INF311 50 60 )

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

49A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java appletsltAPPLET

code = AppletINF311classwidth = 500height = 300gt

ltAPPLETgt

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

50A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

Java applets for online demos

Two technologies in use Image segmentation Texture synthesis (hole filling)

Try it httpwwwsonycslcojppersonnielsenClickRemovalF Nielsen R Nock ClickRemoval interactive pinpoint image object removal ACM Multimedia 2005

Hallucination (mirage)

51A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

There is much more in Java

JDKTM 6 Documentation

52A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

A glimpse at object inheritanceAll objects inherit from the topmost object Objectmeaning some methods are already predefined

Can overwrite themethod toString

53A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

A glimpse at object inheritanceObject

public String toString()(superclass)

Point2Dpublic String toString()

class Point2Ddouble xy ConstructorPoint2D(double xi double yi)thisx=xithisy=yi Overrides default methodpublic String toString()return [+x+ +y+]

class SuperClasspublic static void main(String [] a) Point2D myPoint=new Point2D(MathPI MathE) Systemoutprintln(Point+myPoint)

54A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

• Diapo 1
• Diapo 2
• Diapo 3
• Diapo 4
• Diapo 5
• Diapo 6
• Diapo 7
• Diapo 8
• Diapo 9
• Diapo 10
• Diapo 11
• Diapo 12
• Diapo 13
• Diapo 14
• Diapo 15
• Diapo 16
• Diapo 17
• Diapo 18
• Diapo 19
• Diapo 20
• Diapo 21
• Diapo 22
• Diapo 23
• Diapo 24
• Diapo 25
• Diapo 26
• Diapo 27
• Diapo 28
• Diapo 29
• Diapo 30
• Diapo 31
• Diapo 32
• Diapo 33
• Diapo 34
• Diapo 35
• Diapo 36
• Diapo 37
• Diapo 38
• Diapo 39
• Diapo 40
• Diapo 41
• Diapo 42
• Diapo 43
• Diapo 44
• Diapo 45
• Diapo 46
• Diapo 47
• Diapo 48
• Diapo 49
• Diapo 50
• Diapo 51
• Diapo 52
• Diapo 53
• Diapo 54

52A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

A glimpse at object inheritanceAll objects inherit from the topmost object Objectmeaning some methods are already predefined

Can overwrite themethod toString

53A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

A glimpse at object inheritanceObject

public String toString()(superclass)

Point2Dpublic String toString()

class Point2Ddouble xy ConstructorPoint2D(double xi double yi)thisx=xithisy=yi Overrides default methodpublic String toString()return [+x+ +y+]

class SuperClasspublic static void main(String [] a) Point2D myPoint=new Point2D(MathPI MathE) Systemoutprintln(Point+myPoint)

54A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

• Diapo 1
• Diapo 2
• Diapo 3
• Diapo 4
• Diapo 5
• Diapo 6
• Diapo 7
• Diapo 8
• Diapo 9
• Diapo 10
• Diapo 11
• Diapo 12
• Diapo 13
• Diapo 14
• Diapo 15
• Diapo 16
• Diapo 17
• Diapo 18
• Diapo 19
• Diapo 20
• Diapo 21
• Diapo 22
• Diapo 23
• Diapo 24
• Diapo 25
• Diapo 26
• Diapo 27
• Diapo 28
• Diapo 29
• Diapo 30
• Diapo 31
• Diapo 32
• Diapo 33
• Diapo 34
• Diapo 35
• Diapo 36
• Diapo 37
• Diapo 38
• Diapo 39
• Diapo 40
• Diapo 41
• Diapo 42
• Diapo 43
• Diapo 44
• Diapo 45
• Diapo 46
• Diapo 47
• Diapo 48
• Diapo 49
• Diapo 50
• Diapo 51
• Diapo 52
• Diapo 53
• Diapo 54

53A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

A glimpse at object inheritanceObject

public String toString()(superclass)

Point2Dpublic String toString()

class Point2Ddouble xy ConstructorPoint2D(double xi double yi)thisx=xithisy=yi Overrides default methodpublic String toString()return [+x+ +y+]

class SuperClasspublic static void main(String [] a) Point2D myPoint=new Point2D(MathPI MathE) Systemoutprintln(Point+myPoint)

54A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

• Diapo 1
• Diapo 2
• Diapo 3
• Diapo 4
• Diapo 5
• Diapo 6
• Diapo 7
• Diapo 8
• Diapo 9
• Diapo 10
• Diapo 11
• Diapo 12
• Diapo 13
• Diapo 14
• Diapo 15
• Diapo 16
• Diapo 17
• Diapo 18
• Diapo 19
• Diapo 20
• Diapo 21
• Diapo 22
• Diapo 23
• Diapo 24
• Diapo 25
• Diapo 26
• Diapo 27
• Diapo 28
• Diapo 29
• Diapo 30
• Diapo 31
• Diapo 32
• Diapo 33
• Diapo 34
• Diapo 35
• Diapo 36
• Diapo 37
• Diapo 38
• Diapo 39
• Diapo 40
• Diapo 41
• Diapo 42
• Diapo 43
• Diapo 44
• Diapo 45
• Diapo 46
• Diapo 47
• Diapo 48
• Diapo 49
• Diapo 50
• Diapo 51
• Diapo 52
• Diapo 53
• Diapo 54

54A Concise and Practical Introduction to Programming Algorithms in Java copy 2009 Frank Nielsen

• Diapo 1
• Diapo 2
• Diapo 3
• Diapo 4
• Diapo 5
• Diapo 6
• Diapo 7
• Diapo 8
• Diapo 9
• Diapo 10
• Diapo 11
• Diapo 12
• Diapo 13
• Diapo 14
• Diapo 15
• Diapo 16
• Diapo 17
• Diapo 18
• Diapo 19
• Diapo 20
• Diapo 21
• Diapo 22
• Diapo 23
• Diapo 24
• Diapo 25
• Diapo 26
• Diapo 27
• Diapo 28
• Diapo 29
• Diapo 30
• Diapo 31
• Diapo 32
• Diapo 33
• Diapo 34
• Diapo 35
• Diapo 36
• Diapo 37
• Diapo 38
• Diapo 39
• Diapo 40
• Diapo 41
• Diapo 42
• Diapo 43
• Diapo 44
• Diapo 45
• Diapo 46
• Diapo 47
• Diapo 48
• Diapo 49
• Diapo 50
• Diapo 51
• Diapo 52
• Diapo 53
• Diapo 54