design and analysis of algorithms – chapter 41 decrease and conquer i dr. ying lu [email protected]...

Design and Analysis of Algorithms – Chapter 4 1

Decrease and Conquer I

Dr. Ying [email protected]

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms


Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides from Most of the lecture notes are based on the slides from

the Textbook’s companion websitethe Textbook’s companion website– http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin

• Some examples and slides are based on lecture notes Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical created by Dr. Ben Choi, Louisiana Technical University, Dr. Chuck Cusack, Hope College and University, Dr. Chuck Cusack, Hope College and İnanç Tahralı, Gebze Institute of Technology İnanç Tahralı, Gebze Institute of Technology

• I have modified many of their slides and added new I have modified many of their slides and added new slides.slides.

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms


Decrease and ConquerDecrease and Conquer

1.1. Reduce a problem instance to a smaller instance Reduce a problem instance to a smaller instance of the same problem and extend solutionof the same problem and extend solution

2.2. Solve the smaller instanceSolve the smaller instance

3.3. Extend solution of smaller instance to obtain Extend solution of smaller instance to obtain solution to original problemsolution to original problem

Also referred to as Also referred to as inductiveinductive,, incremental incremental approach or approach or chip and conquerchip and conquer


Examples of Decrease and Examples of Decrease and ConquerConquer

Decrease by one:Decrease by one:• Insertion sortInsertion sort• Graph algorithm:Graph algorithm:

– Topological sortingTopological sorting• Algorithms for generating permutations, subsetsAlgorithms for generating permutations, subsets

Decrease by a constant factorDecrease by a constant factor• Binary search Binary search • Fake-coin problemsFake-coin problems• multiplication à la russemultiplication à la russe• Josephus problemJosephus problem

Variable-size decreaseVariable-size decrease• Euclid’s algorithmEuclid’s algorithm• Selection by partitionSelection by partition

Insertion Sort

A Decrease-by-One algorithm:A Decrease-by-One algorithm:

Insertion-Sort(A[0…n-1]) {Insertion-Sort(A[0…n-1]) {

Insertion-Sort(A[0…n-2])Insertion-Sort(A[0…n-2])

Insert A[n-1] into the sorted list A[0…n-2]Insert A[n-1] into the sorted list A[0…n-2]

return the sorted list A[0…n-1]return the sorted list A[0…n-1]

}}


Directed Acyclic GraphsDirected Acyclic Graphs

A A directed acyclic graphdirected acyclic graph or or DAGDAG is a directed graph is a directed graph with no directed cycles:with no directed cycles:


Topological Sort of a DAG (I)Topological Sort of a DAG (I)

DAGs arise in many modeling problems, DAGs arise in many modeling problems, e.g.:e.g.:• course prerequisite structurecourse prerequisite structure• food chainsfood chains

A topological sort implies a partial ordering A topological sort implies a partial ordering on the domainon the domain


Topological Sort of a DAG (II)Topological Sort of a DAG (II)

Topological sortTopological sort of a DAG: of a DAG:• Linear ordering of all vertices in graph G Linear ordering of all vertices in graph G such that for every edge (such that for every edge (uu, , vv) ) G, the vertex, G, the vertex,

u, where the edge starts is listed before the u, where the edge starts is listed before the vertex, v, where the edge ends.vertex, v, where the edge ends.

cs101

cs203

math101

cs310

cs401

cs445

cs525

cs990


Topological SortTopological Sort

cs101 cs203 math101 cs310 cs401 cs445 cs525 cs990


Topological Sorting Algorithm Topological Sorting Algorithm (I)(I)

1. Source removal algorithm1. Source removal algorithm• Repeatedly identify and remove a Repeatedly identify and remove a sourcesource vertex, i.e., a vertex, i.e., a

vertex that has no incoming edgesvertex that has no incoming edges

• How would you find a source (or determine that such a How would you find a source (or determine that such a vertex does not exist) in a digraphvertex does not exist) in a digraph– represented by adjacency matrix?represented by adjacency matrix?

– represented by adjacency linked list?represented by adjacency linked list?



1. Source removal algorithm 1. Source removal algorithm • Repeatedly identify and remove a Repeatedly identify and remove a sourcesource vertex, i.e., a vertex, i.e., a


• How would you find a source (or determine that such a How would you find a source (or determine that such a vertex does not exist) in a digraphvertex does not exist) in a digraph– represented by adjacency matrix?represented by adjacency matrix?

– represented by adjacency linked list?represented by adjacency linked list?

• ΘΘ(|(|V|V|22) using adjacency matrix) using adjacency matrix

• ΘΘ(|(|V|V|+|E|) using adjacency linked lists+|E|) using adjacency linked lists



1. Source removal (a Decrease-by-One) algorithm1. Source removal (a Decrease-by-One) algorithm• Repeatedly identify and remove a Repeatedly identify and remove a sourcesource vertex, i.e., a vertex, i.e., a




Topological Sorting Algorithm Topological Sorting Algorithm (II)(II)

2. DFS-based algorithm:2. DFS-based algorithm:

cs101

cs203

math101

cs310

cs401

cs445

cs525

cs990


Topological Sorting Algorithm Topological Sorting Algorithm (II)(II)

2. DFS-based algorithm:2. DFS-based algorithm:

• DFS traversal noting order vertices are popped off stackDFS traversal noting order vertices are popped off stack

• Reverse order solves topological sortingReverse order solves topological sorting

• ΘΘ((VV22) using adjacency matrix) using adjacency matrix

• ΘΘ(V+E) using adjacency linked lists(V+E) using adjacency linked lists

In-Class ExerciseIn-Class Exercise

Page 143Page 143


Sum RuleSum Rule

If a task can be done either in one of nIf a task can be done either in one of n11 ways or in ways or in

one of none of n22 ways, where none of the set of n ways, where none of the set of n11 ways is ways is

the same as any of the set of nthe same as any of the set of n22 ways, then there are ways, then there are

nn11+n+n22 ways to do the task ways to do the task

In general,In general, If AIf A11, A, A22, …, A, …, Amm are disjoint finite sets, then the are disjoint finite sets, then the

number of elements in the union of these sets is as number of elements in the union of these sets is as followsfollows

|A|A11⋃A⋃A22 ⋃… ⋃A ⋃… ⋃Amm|=|A|=|A11|+|A|+|A22|+…+|A|+…+|Amm||Design and Analysis of Algorithms – Chapter 4 16

Sum RuleSum Rule

If AIf A11, A, A22, …, A, …, Amm are disjoint finite sets, then the are disjoint finite sets, then the

number of elements in the union of these sets is as number of elements in the union of these sets is as followsfollows

|A|A11⋃A⋃A22 ⋃… ⋃A ⋃… ⋃Amm|=|A|=|A11|+|A|+|A22|+…+|A|+…+|Amm||


Insertion Sort

A Decrease-by-One algorithm:A Decrease-by-One algorithm:

Insertion-Sort(A[0…n-1]) {Insertion-Sort(A[0…n-1]) {

Insertion-Sort(A[0…n-2])Insertion-Sort(A[0…n-2])

Insert A[n-1] into the sorted list A[0…n-2]Insert A[n-1] into the sorted list A[0…n-2]

return the sorted list A[0…n-1]return the sorted list A[0…n-1]

}}


Topological Sorting AlgorithmTopological Sorting Algorithm

Source removal (a Decrease-by-One) algorithmSource removal (a Decrease-by-One) algorithm• Repeatedly identify and remove a Repeatedly identify and remove a sourcesource vertex, i.e., a vertex, i.e., a



Design and Analysis of Algorithms - Chapter 4 20

Algorithms for Generating Combinatorial Objects

Definition : Most important types of combinatorial objects

permutations combinations subsets of a given set

They typically arise in problems that require a consideration of different choices TSP, Knapsack

To solve these problems need to generate combinatorial objects


Generating Permutations

Assume the set whose elements need to be permuted is the set of integers from 1 to n They can be interpreted as indices of elements in

an n-element set {ai, …, an}



Assume the set whose elements need to be permuted is the set of integers from 1 to n They can be interpreted as indices of elements in

an n-element set {ai, …, an}

What would the decrease-by-one technique suggest for the problem of generating all n! permutations?



Approach : The smaller-by-one problem is to generate all (n-

1)! permutations Assuming that the smaller problem is solved We can get a solution to the larger one

by inserting n in each of the n possible positions among elements of every permutation of n-1 elements

Total number of all permutations will be n.(n-1) ! = n!


Generating Permutations We can insert n in the previously generated

permutations left to right right to left


Generating Permutations minimal-change requirement

Each permutation differs from the previous one by exchanging only two elements

Benefical for algorithm’s speed EX: Advantage of this order in TSP:

The length of the new tour can be calculated in constant time By using the length of the previous tour Tour Cost

a-b-c-d-e-f-g-h-a 2+4+3+5+2+1+4+1=22

a-b-c-d-e-f-h-g-a 2+4+3+5+2+3+4+2=25



Approach : Assume all permutations of {1,2,…,(n-1)} are available Start with inserting n into 1,2,…,(n-1) by moving right to left Then swicth direction every time a new permutation of {1,2,…,

(n-1)} is processed


Generating Permutations This order satisfies minimal-change requirement

This approach requires that all the permutations of {1,2,…,(n-1)} are calculated already Not easy to do!... (requires lots of space)



Another way to get the same ordering : Associate a direction with each component k in a

permutation Indicate such a direction by a small arrow

3 2 4 1 The component k is said to be mobile

if its arrow points to a smaller number adjacent to it



Another way to get the same ordering : Associate a direction with each component k in a

permutation Indicate such a direction by a small arrow

3 2 4 1 The component k is said to be mobile

if its arrow points to a smaller number adjacent to it 3 and 4 are mobile 2 and 1 are not

The following algorithm uses this notion


Generating PermutationsALGORITHM Johnson Trotter (n) // Implements Johnson-Trotter algorithm for generating

permutations // Input : A positive integer n // Output : A list of permutations of {1, … , n} Initialize the first permitation with 1 2 … n while there exists a mobile integer k do find the largest mobile integer k

swap k and the adjacent integer its arrow points to reverse the direction of all integers that are larger than k



An application of Johnson Trotter algorithm :

1 2 3 1 3 2 3 1 2 3 2 1 2 3 1 2 1 3


In-class ExerciseIn-class Exercise

Page 148 exercise 4.3.2Page 148 exercise 4.3.2 Generate all permutations of {1,2,3,4} byGenerate all permutations of {1,2,3,4} by

a. the bottom-up minimal-change algorithm.a. the bottom-up minimal-change algorithm.

b. the Johnson-Trotter algorithm.b. the Johnson-Trotter algorithm.


Generating PermutationsGenerating Permutations

The Johnson-Trotter algorithm does not produce The Johnson-Trotter algorithm does not produce permutations in lexicographical orderpermutations in lexicographical order

ExampleExample Johnson-Trotter algorithm: 123, 132, 312, 321, 231, Johnson-Trotter algorithm: 123, 132, 312, 321, 231,

213213 Lexicographical order: 123, 132, 213, 231, 312, 321Lexicographical order: 123, 132, 213, 231, 312, 321

Use an example: produce permutations {1,2,3,4,5} in Use an example: produce permutations {1,2,3,4,5} in lexicographical order, to get an idea for the algorithmlexicographical order, to get an idea for the algorithm

design and analysis of algorithms – chapter 41 decrease and conquer i dr. ying lu [email protected]...

Documents

analysis of algorithms

source vertex

topological sortingalgorithms

directed graph

insertion sortgraph

dag iitopological sort

examples of decrease

insertion sorta decrease