ApplicationsData Structures and Algorithms (60-254)

2

Encoding and Decoding Messages

3

Fixed-length coding

Code for each character has the same number of bits.

QuestionHow many bits do we need to encode uniquely each character in a message made up of characters from an n-letter alphabet? Answer log n bits at least.

4

Variable-length coding

5

Another encoding scheme

6

Prefix codes

Code word of a character is NOT prefix of the code for a character , where .Also, called prefix-free because the term “prefix” is misleading… Meaning of prefixExamples:

00 is prefix of 001110 is prefix of 110111111101 is not prefix of 1111001

7

What is interesting in a prefix code?

8

Lossless Data Compression

M = a d d d

Encoding using scheme 1

C1= 000111Encoding using scheme 2

C2=00111111

Length(C1)=6

Length(C2)=8 Lossless data compression is possible using prefix codes

9

Huffman coding

10

Huffman coding

11

Huffman coding

12

Huffman coding

13

Huffman coding

14

Huffman coding

15

Huffman coding

16

Huffman coding

17

Huffman coding

18

Huffman coding

19

Graphs

20

Graphs

We denote a graph by G=(V, E) where V = Set of vertices andE = Set of edges

|V| = Number of vertices and|E| = Number of edges

G is directed if the edges are, otherwise G is undirected.

21

Graph representation

22

Graph representation

23

Graph Representation

24

Breadth-first search

Also called: level order search

What does it do? Given: G=(V, E), a directed or undirected graph and a “source” vertex s V It systematically explores vertices reachable from s, to construct a breadth-first search tree

25

Shortest Path

Length of the path from s to each reachable vertex is the shortest path from s to this vertex Example graph

26

Expand node s

27

Expand node w

28

Expand node r

29

Expand node t

30

Expand node x

31

Expand node v

32

Expand node u

33

Breath-first Search Tree

34

Formal Algorithm

Colour terminology:Gray nodes make up the “frontier” (red line) White nodes (vertices) are the unexplored ones Black nodes are the completely explored ones, lying on the other side of the “frontier”

35

Breadth-first search algorithm

BFS(G, s) For each vertex uV –{s} { // Setup loop O(|V|)

u.color WHITE;u.distance ;u.parent NULL; // parent of u

}s.color GRAY;s.distance 0;s.parent NULL;Q Q {s};

(Cont’d.)

36

Breadth-first search algorithm

While (Q ) { // Processing loop O(|E|)u head (Q);for each v adjacent to u {

if (v.color = WHITE) {v.color GRAY;v.distance u.distance+1;v.parent u;Q Q {v};

}}Delete u from Q;u.color BLACK;

}

37

Analysis

Thm. Running time of BFS is O (|V|+|E|) Pf Queuing operations take O (|V|) time.Each edge in the graph is looked at most once. Hence O (|E|) time and the theorem follows.

38

We can prove the following factsFact1:Breath-first-search (BFS) discovers every

vertex reachable from s Fact2:The length of the path from s to a reachable

vertex v, is a shortest path from s to v.

39

Weighted Graph

Each edge has a weight (a number).

40

Minimum Spanning Tree (MST)

Green edges form a minimum spanning tree of the example graph

41

Minimum Spanning Tree

Weight of the MST is: W(T) = 8 + 2 + 4 + 7 + 4 + 2 + 9 + 1 = 37

42

Formally

Given a connected, weighed, undirected graph G=(V, E), Find an acyclic subset TE that connects all the vertices such that

is minimum, where weight(u,v) is the weight of the edges connecting u to v.

43

Growing an MST, greedily

Let A be a subset of edges of some MST

Def. An edge e=(u,v) of G is safe for A if A {(u,v)} is also a subset of an MST.

44

A generic MST algorithm

GENERIC_MST(G,w) A ;

while (A does not form an MST)

find a safe edge (u,v) for A; A A {(u,v)};

return A;

45

Cuts and safe edges

Definition: A cut of G is a partition (S, V-S) of V

46

A few definitions

Definition: An edge (u,v) E crosses the cut (S,V-S) if one of its end points is in S and the other in V-S Definition: A cut respects the set A if no edge in A crosses the cut Definition: An edge is a light edge crossing a cut if its weight is the minimum of any edge crossing the cut.

47

Safe Edges

The following theorem characterizes a safe edge for A TheoremLet A be a subset of E that is included in some MST for G. Let (S,V-S) be any cut of G that respects A, and Let (u,v) be a light edge crossing (S,V-S). Then, edge (u,v) is safe for A.

48

Prim’s algorithm

Select vertex aS={a}

49

(a, b) is a light edge for (S, V-S)

Select vertex bnew S={a, b}

50

(b,c) is a light edge for (S,V-S)

Select vertex cnew S={a, b, c}

51

(c, i) is a light edge for (S,V-S) where S={a, b, c}Select vertex inew S={a, b, c, i}

52

(c, f) is a light edge for (S,V-S)

Select vertex fnew S={a, b, c, f, i}

53

(f, g) is a light edge for (S,V-S)

Select vertex gnew S={a, b, c, g, f, i}

54

(g, h) is a light edge for (S,V-S)

Select vertex hnew V-S={d, e}

55

(c, d) is a light edge for (S,V-S)

Select vertex dnew V-S={e}

56

(d, e) is a light edge for (S,V-S)

Select vertex enew V-S=

57

Formal Algorithm

MST_Prim(G, W, r) QV;for (each u Q)

u.key ;

r.key 0;r.parent NULL;while ( Q )

u extract_min(Q);for (each v adjacent to u) if v Q and weight(u,v) < v.keyv.parent u;v.key weight(u,v)

58

Important Note on Shortest Path: Dijkstra’s algorithmThe algorithm incrementally constructs a set S of vertices to which the shortest path is known.

It adds a vertex v (from the remaining vertices) to S,whose distance from S (source) is

the shortest of the remaining vertices. The assumption of non-negative weights guarantees that the shortest path to v passes only through vertices in S.

59

Priority Queues

Let S be a set of keys, a priority queue supports the following operations on S: 1. Insert (S, x)2. Minimum (S) -- returns min key3. Extract_Min (S) -- returns and removes min key

60

Heap as Priority Queue

A priority queue can be implemented using a heap A heap is a complete binary tree, satisfying the heap property:

For a node v,v.key v.left.keyv.key v.right.key

61

Single-source shortest paths

Let G be a weighted, directed graph.

A path in the graph from u to v is a sequence of edges that connects u to v. The weight of this path is the sum of the weights of its constituent edges. Let (u, v) denote a minimum weight path from u to v Let s be a source vertex of G

62

Single-source shortest path problemCompute a shortest path from s to each vertex v. Remark: A shortest path is a path of minimum weight. Dijkstra’s algorithm:

We assume that the weights of the edges are non-negative.

63

Relaxation

64

Relax

Relax(u,v,w) {If (v.distance > u.distance +w(u,v)) {

v.distance u.distance +w(u,v);v.parent u;

}}

Initialize_Simple_Source (G, v) {For each vertex v V [G];v.distance ;v.parent NULL;}

65

Dijkstra’s algorithm by example

The algorithm maintains a set X of vertices whose final shortest-path weights from s have already been determined

66

Add s to X

Relax vertices adjacent to s

67

Add x to X

Relax vertices adjacent to x

68

Add y to X

Relax vertices adjacent to y

69

Add u to X

Relax vertices adjacent to u

70

Add v to X

Done

71

Formal Algorithm

Dijkstra(G,w,s) Initialize_single_source(G,s)

XQV [G]while (Q )

uextract_min(Q)XX{u}For each vertex v adjacent to u

Relax(u,v,w)