meljun cortes algorithm greedy algorithm design technique i

8/21/2019 MELJUN CORTES ALGORITHM Greedy Algorithm Design Technique I

1/13

Design and Analysis of Algorithms

Greedy Algorithm Design Technique *Property o f STIPage 1 of 13

TOPIC TITLE: Greedy Algorithm Design Technique

Speci f ic Object ives:

At the end of the topic session, the students are expected to:

Cognitive:

1. Understand the concept behind Greedy algorithm designtechnique.

2. Understand the application of Prim’s algorithm design technique.3. Study the application of Kruskal’s algorithm design technique.4. Understand the application of Dijsktra’s algorithm. 5. Familiarize with the application of Huffman trees.

Affective:

1. Listen to others with respect.2. Participate in class discussions actively.

3. Share ideas to the class.

MATERIALS/EQUIPMENT:

o topic slides

o OHP

TOPIC PREPARATION:

o Have the students research on the following: Definition of Greedy algorithm design technique Types of Greedy algorithm design technique

o Provide sample problems that can be solved using Greedyalgorithm design technique.

o It is imperative for the instructor to incorporate various kinds ofteaching strategies while discussing the suggested topics. Theinstructor may use the suggested learning activities below tofacilitat e a thorough and creative discussion of the topic.

o Prepare the slides to be presented in the class.o Prepare seatwork for the students to apply the lessons learned

in Greedy algorithm design technique.

TOPIC PRESENTATION:

The topic will cover the application of Greedy algorithm designtechnique.

The following is the suggested flow of discussion for the course topic:

1. Call on students and ask the definition of Greedy algorithmdesign technique.

2. Enumerate the importance of Greedy algorithm designtechnique.

3. Discuss the application of Greedy algorithm design technique inthe following:

a. Prim’s algorithm b. Kruskal’s algorithm c. Dijktra’s algorithm

d. Huffman trees algorithm4. Simulate sample problems using Greedy algorithm designtechnique.


2/13



Greedy Algorithm DesignTechniquePage 1 of 25

Greedy Algorithm Design Technique

The following are the topics to be discussed under Space and TimeTradeoffs topic:

o Definition of Greedy algorithm

o Importance of Greedy algorithm design techniqueo Types of Greedy algorithm

o Prim’s algorithm o Kruskal’s algorithm o Dijktra’s algorithm o Huffman trees

o Sample problem using the application of Prim’s algorithm,Kruskal’s algorithm, Dijktra’s algorithm, and Huffman trees.

[Greedy Algorithm Design Technique, Page 1 of 25]

About Greedy Algorithm DesignTechniquePage 2 of 25

Page 3 of 25

About Greedy Algorithm Design Technique

Greedy algorithms are simple and straightforward type. They areshortsighted in their approach in the sense that they take decisions onthe basis of information at hand without worrying about the effect thesedecisions may have in the future [www.personal.kent.edu].

Most greedy algorithms are easy to develop, implement, and timeefficient. There are many problems that cannot be solved correctly bygreedy approach, but it is being used to solve optimization problems.Computer scientists consider the greedy approach as a general designtechnique despite the fact that it is applicable to optimization problemsonly. The greedy approach proposes the construction of solution in aseries of steps, each expanding a partially constructed solution obtainedso far, until a complete solution to the problem is reached. On eachstep, which is the central point of this technique, the choices made mustbe:

Feasible – it has to satisfy the problem’s constraints

Locally optimal – it has to be the best local choice among allfeasible choices available on that step

Irrevocable – once made, it cannot be change on subsequentsteps of the algorithm

These requirements explain the Greedy algorithm techniques, which oneach step suggests a “greedy” grab of the best alternative in the desirethat a sequence of locally optimal choices that will yield a globallyoptimal solution to the entire problem. From algorithmic perspective, thequestion is whether a greedy strategy works or not. There are problemsfor which a sequence of locally optimal choices does yield an optimalsolution for every instance of the given problem.

To better understand the algorithm, consider the example about “MakingChange” problem. Consisting of available dollar coins (100 cents),quarters (25 cents), dimes (10 cents), nickels (5 cents), and pennies (1cent), make a change of a given amount using the smallest possible

number of coins. To solve this problem, you can use simple and formal

algorithms as follows:


3/13



Page 4 of 25

Page 5 of 25

Page 6 of 25

1. Informal Algorithm – starts with nothing and at every stagewithout passing the given amount and adds the largest coinsalready chosen.

2. Formal Algorithm – makes change for n units using the leastpossible number of coins.

Figure 12.1 Greedy algorithm

For example, make a change for 2.89 (289 cents) where n = 2.89 andthe solution contains 2 dollars, 3 quarters, 1 dime, and 4 pennies. Thealgorithm is greedy because at every stage, it chooses the largest coinwithout worrying about the consequences.

The greedy algorithm consists of four functions and these are as follows:

1. A function that validates whether the selected set of itemsprovides a solution.

2. A function that validates the feasibility of a set.3. The selection function that tells which of the choices is the most

capable.

4. The objective function which does not appear explicitly, butgives the value of a solution.

[About Greedy Algorithm Design Technique, Pages 2-6 of 25]

MAKE-CHANGE (n)C ← {100, 25, 10, 5, 1} // constantSol ← {}; // set that will hold the solution setSum ← 0 sum of item in solution set WHILE sum not = n

x = largest item in set C such that sum + x ≤ n IF no such item THEN

RETURN "No Solution"S ← S {value of x} sum ← sum + x

RETURN S


4/13



Prim’s Algorithm Page 7 of 25

Page 8 of 25

Page 9 of 25

Prim’s Algorithm

Prim's algorithm is an algorithm in graph theory that finds a minimumspanning tree for a connected weighted graph. This means it finds asubset of the edges that forms a tree that includes every vertex, wherethe total weight of all the edges in the tree is minimized. If the graph is

not connected, then it will only find a minimum spanning tree for one ofthe connected components [www.wordiq.com].

To better understand Prim’s algorithm, let us examine a sample problemgiven the n points then connect them in the best possible way to allocatea path between pair of points. Then represent the points by vertices of agraph, possible connections by the graphs edges and the connectioncosts by the edge weights. The questions can be posed as theminimum spanning tree problem that defines the following:

A spanning tree of a connected graph is its connected acyclicsubgraph like a tree that contains all the vertices of the graph[LEV07].

A minimum spanning tree (MST) of a weighted connected graphis its spanning tree of the smallest weight, where the weight of atree is defined as the sum of the weights on all its edges[LEV07].

The minimum spanning tree problem is the problem of finding aminimum spanning tree for a given weighted connected graph[LEV07].

Prim’s algorithm works as follows:

Figure 12.2 Prim’s algorithm

Prim’s algorithm creates a MST through a sequence of expandingsubtrees. The initial subtree in such a sequence consists of a singlevertex selected arbitrarily from the set V of the graphs vertices. You

then expand the current tree in the greedy manner by simply attaching toit the nearest vertex not in the tree in each repetition.

Finally, the algorithm will be terminated after all the vertices in graph areincluded in the tree. Since the algorithm expands the three by exactlyone vertex on each repetition is n-1, where n is the number of vertices inthe graph. The generated tree is obtained as the set of edges used forthe expanding of the tree.

1. Create a tree containing a single vertex, chosenarbitrarily from the graph

2. Create a set containing all the edges in the graph3. Loop until every edge in the set connects two vertices inthe tree

a. Remove from the set an edge with minimumweight that connects a vertex in the tree with avertex not in the tree

b. Add that edge to the tree


5/13



Page 10 of 25 The illustration below is an example of minimum spanning tree.

Figure 12.3 Graph

The figure illustrates a graph having four nodes with its equivalent cost.Based on the graph, you can come up with the illustration below for theminimum spanning tree:

Figure 12.4 Minimum spanning tree

The figure shows the minimum spanning tree from the graph givenabove.

Prim’s algorithm efficiency:

O(n2 ) for weight matrix representation of graph and arrayimplementation of priority queue

O(m log n) for adjacency lists representation of graph with n vertices and m edges and min-heap implementation of thepriority queue

[Prim’s Algorithm, Pages 7-10 of 25]

Kruskal’s Algorit hmPage 11 of 25

Kruskal’s Algorithm

Kruskal's algorithm is an algorithm in graph theory that finds a minimumspanning tree for a connected weighted graph [www.wordiq.com].

It was introduced by Joseph Kruskal and is an example of greedyalgorithm. This is an algorithm which finds a subset of the edges thatforms a tree that includes every vertex, where the total weight of all theedges in the tree is minimized. If in case the graph is not connected, itfinds a minimum spanning forest connected component.


6/13



Page 12 of 25

Page 13 of 25

Page 14 of 25

Kruskal’s algorithm works as follows:

Figure 12.5 Kruskal’s algorithm

Each edge in the ordered sequence is checked to see whether it can beconsidered part of the tree. It actually looks at the minimum spanningtree for a weighted connected graph G = {V, E} as an acyclic sub-graphwith |V| - 1 edges for which the sum of the edge weights is the smallest.

Consequently, the algorithm constructs a minimum tree as an expandingsequence of sub-graphs, which are always acyclic but are notnecessarily connected on the intermediate stages of the algorithm.

The algorithm begins by sorting the graph’s edges in non-decreasingorder of their weights. Starting with the empty sub-graph, it scans thissorted list adding the next edge on the list to the current sub-graph ifsuch an inclusion does not create a cycle and simply skipping the edgeotherwise.

To further understand Kruskal’s algorithm, let us examine a step by stepexample of its application.

Step 1. Observe that in the given graph, the edge (g, h) is the shortest.In this case, either vertex g or vertex h could be represented. Letschoose vertex subjectively:

Figure 12.6 Illustration of Step 1

Step 2. The edge (c, i) creates the second tree. Choose vertex c asrepresentative for second tree.


1. Create a forest F (a set of trees), where each vertex in

the graph is a separate tree

2. Create a set S containing all the edges in the graph

3. While S is nonemptya. Remove an edge with minimum weight from S

b. If that edge connects two different trees, thenadd it to the forest, combining two trees into a

single tree

c. Otherwise discard that edge


7/13



Page 15 of 25

Page 16 of 25

Page 17 of 25

Step 3. Edge (g, g) is the next shortest edge. Add this edge and choosevertex g as representative.


Step 4. Edge (a, b) creates a third tree.


Step 5. Add edge (c, f) and merge two trees. Vertex c is chosen as therepresentative.


Step 6. Edge (g, i) is the next cheapest, but if we add this edge a cyclewould be created. Vertex c is the representative of both.


Step 7. Instead, add edge (c, d).



8/13



Step 8. If we add edge (h, i), this would make a cycle.


Step 9. Instead of adding edge (h, i) add edge (a, h).


Step 10. Again, if we add edge (b, c), it would create a cycle. Add edge

(d, e) instead to complete the spanning tree. In this spanning tree, alltrees joined and vertex c is a sole representative.


[Kruskal’s Algorithm, Pages 11-17 of 25]

Dijktra’s Algorithm Page 18 of 25

Dijktra’s Algorithm

Dijktra’s algorithm was conceived by Dutch computer scientist EdsgerDijktra in 1959. It is a graph search algorithm that solves the single-

source shortest path problem for a graph with nonnegative edge pathcosts, producing a shortest path tree [www.wikipedia.org].

Dijktra’s algorithm finds the shortest path to a graph’s vertices in order oftheir distance from a given source. It finds the first shortest path fromthe source to a vertex nearest to it, then to the second and so on. Ingeneral, before its i th iteration commences, the algorithm has alreadyidentified the shortest paths to i-1 other vertices nearest to the source.

To facilitate the algorithm’s operations, we label each vertex with twolabels. The numeric label d indicates the length of the shortest path

from the source to the vertex found by the algorithm so far.

http://en.wikipedia.org/wiki/Computer_scientisthttp://en.wikipedia.org/wiki/Computer_scientisthttp://en.wikipedia.org/wiki/Graph_search_algorithmhttp://en.wikipedia.org/wiki/Shortest_path_problemhttp://en.wikipedia.org/wiki/Graph_(mathematics)http://en.wikipedia.org/wiki/Edge_(graph_theory)http://en.wikipedia.org/wiki/Shortest_path_treehttp://en.wikipedia.org/wiki/Shortest_path_treehttp://en.wikipedia.org/wiki/Edge_(graph_theory)http://en.wikipedia.org/wiki/Graph_(mathematics)http://en.wikipedia.org/wiki/Shortest_path_problemhttp://en.wikipedia.org/wiki/Graph_search_algorithmhttp://en.wikipedia.org/wiki/Computer_scientist


9/13



Page 19 of 25 The algorithm repeats the following steps until all vertices areprocessed:

Figure 12.16 Dijktra’s algorithm

[Dijktra’s Algorithm, Pages 18-19 of 25]

Huffman Tree AlgorithmPage 20 of 25

Page 21 of 25

Huffman Tree Algorithm

A Huffman tree is a binary tree which is based on the probabilitydistribution of a symbol set and the principle that symbols occurringmore frequently will be represented by shorter codes than other, lessprobable ones. [www.cs.mcgill.ca]

Due to this principle Huffman coding represents a powerful method ofdata compression, provided that the symbol probabilities are accuratelyestimated. Before we can assign bit patterns to each character, we willassign each character a weight based on its frequency of use. In ourexample, we assume that the frequency of the character E in a text is15% and the frequency of the character T is 12%, as illustrated in the

table below:

E = 15 T = 12 A = 10 O = 08 R = 07 N = 06 S = 05

U = 05 I = 04 D = 04 M = 03 C = 03 G = 02 K = 02

Table 12.1 Character weights for a sample of Huffman code

Since we have already determined the weight of each character, we cannow build a tree based on its assigned values. The common practice inconstructing a tree follows the three basic steps below:

Step 1: Arrange the entire character set into a row and should beordered according to its frequency from lowest to highest or vice versa.

As a result, every character is now considered as a node at the leaf levelof a tree.

Step 2: Subsequently, we will look for the two nodes with the smallest

combined frequency weights and combine them to form a third node,ensuing a simple two-level tree. Therefore, the weight of the new nodeis based on the combined weight of the original two nodes. We cansimply say that this node, which is one level up from both leaves, iseligible to be combined with other nodes. Keep in mind that, the sum ofthe weights of the two nodes chosen must be smaller than thecombination of any other possible choices.

Step 3: Repeat Step 2 until all of the nodes on each level are combined

in a single tree.

1. From the set of vertices for which kv= false, select the vertex vhaving the smallest known distance dv.

2. Set kv to true.3. For each vertex w adjacent to v for which kv is false, test if the

known distance, dw is greater than dv + C(v, w). If it is, set dw to dv + C(v, w) and set pw to v.


10/13



Page 22 of 25

Page 23 of 25

Page 24 of 25

The figure below illustrates the first part of the Huffman tree derived fromthe character weights.

Figure 12.17 Huffman tree - part 1

The given figure illustrates the output of the three basic steps inconstructing a tree. The first row of the given figure shows Step 1, withthe leaf-level nodes representing the original characters are arranged indescending order of value. The second row shows the Step 2 where welocate the two nodes with the smallest values and combine them. As youobserve, this process results in the creation of a new node that isrepresented by a solid circle. The frequency value (weight) of the newnode is the sum of the weight of the two nodes. Finally, Step 3 isillustrated in the 3

rd row where we combine two or more nodes and apply

it to the rest of the nodes. In the 6th row, the nodes with the lowest value

are one level up from the characters and combine them into a node twolevels up from the leaves.

In the last row, the lowest value node is 08(0) and the second is thevalue 10(A) and there are three 10’s , one at the leaf of level (A), secondis at the level (S,U), and third is two levels up from the leaves (M,C,G,K).Now, the question is which among the lowest values are we going tochoose? The answer is we choose which ever of the 10’s value isadjacent to the 8. This decision keeps the branch lines from crossingand allows us to preserve the legibility of the tree.


11/13



Page 25 of 25 If in case, none of the higher values are adjacent to the lower value, wecan rearrange the nodes for clarity, as shown in the figure below:

Figure12.18 Huffman tree – part 2

In the given figure, the third row shows that characters T and E havebeen moved from the left side to the right of the tree to combine it with anode on that side.

Therefore, we can say that the rest of the process completed the tree

resulting to a single node at the root level with the value of 86, as shownin the 3

rd row of the following figure:

Figure12.19 Huffman tree – part 3


12/13



The figure below shows the rest of the process. As you observed, thecompleted tree results to a single code in the root level having the valueof 86. Once a tree is complete, we use it to assign codes for eachcharacter. To assign codes to each character, starting from the root, weassign 0 to the left branch and 1 to the right branch then do the sameprocess to the other nodes.

Figure 12.20 Huffman tree – final part

A = 100 U = 0101 E = 100 M = 11000 K = 11011

O = 001 I = 0110 R = 1010 C = 11001 T = 111

S = 0100 D = 0111 N = 1011 G = 11010Table 12.2 Character weights for a sample of Huffman code

Huffman trees algorithm:

Figure 12.21 Huffman’s algorithm

The figure above illustrates the simulation of searching algorithm. Itstarts from comparing the first element 24 to the target element 9. Since

the value is not equal to 9, the comparison will proceed to the nextelement, which is 46. This is still not equal to 79, which happens to bethe next element. Finally, the target is found at the next element, whichis 9, and is equal to 9.

[Huffman Tree Algorithm, Pages 20-25 of 25]

GENERALIZATION:

o Greedy algorithms are simple and straightforward type. Theseare shortsighted in their approach in the sense that they takedecisions on the basis of information at hand without worryingabout the effect these decisions may have in the future.

o Prim's algorithm is an algorithm in graph theory that finds aminimum spanning tree for a connected weighted graph.

1. Place each symbol in leaf:Weight of leaf = symbol frequency

2. Select two trees L and R (initially leafs), such that L, Rhave lowest frequencies in tree.

3. Create new (internal) node:

Left child ⇒L

Right child ⇒ R

New frequency ⇒ frequency( L ) + frequency( R)

4. Repeat until all nodes merged into one tree.


13/13



Meaning, it finds a subset of the edges that forms a tree thatincludes every vertex, where the total weight of all the edges inthe tree is minimized. If the graph is not connected, then it willonly find a minimum spanning tree for one of the connectedcomponents.

o Kruskal's algorithm is an algorithm in graph theory that finds a

minimum spanning tree for a connected weighted graph.o Dijktra’s algorithm is a graph search algorithm that solves the

single-source shortest path problem for a graph with

nonnegative edge path costs, producing a shortest path tree. o Huffman tree are binary tree which are based on the probability

distribution of a symbol set and the principle that symbolsoccurring more frequently will be represented by shorter codesthan the other, less probable ones.

REFERENCES:

http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgorithms/GraphAlgor/kruskalAlgor.htm

www.wordiq.com http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgorit

hms/Greedy/greedyIntro.htm Anany Levitin,(2007), The design and analysis of algorithm (2nd

ed.), Pearson Education Inc.

meljun cortes algorithm greedy algorithm design technique i

Documents