ads unit 3 ppt

UNIT IIIDYNAMIC PROGRAMMING ALGORITHMS

Developing a Dynamic Programming Algorithm-Subtle Points- Question for the Little Bird- Sub instances and Sub solutions-Set of Substances-Decreasing Time and Space-Number of Solutions-Code. Reductions and NP-Completeness-Satisfiability-Proving NP-Completeness- 3-Coloring- Bipartite Matching. Randomized Algorithms-Randomness to Hide Worst Cases- Optimization Problems with a Random Structure.

Dynamic Programming• A hard topic.• I try to provide a unified way to think of it

and a fixed set of steps to follow.• Even if you don’t get the details of the

algorithm correct, at least get the right structure. • I provide analogies (little bird) to make it

hopefully more fun & easier to follow.

•An optimization problem•Each solution is either valid or not (no cost)•The output is • Yes, it has a valid solution.• No, it does not• the solution is not returned

•Eg: Given graph and integer <G,k>, does G have a clique of size k?

Non-DeterministicPoly-Time Decision Problems (NP)

•Key: Given • an instance I (= <G,k>)• and a solution S (= subset of nodes)• there is a poly-time alg Valid(I,S) to test

whether or not S is a valid solution for I.• Poly-time in |I| not in |S|. |S| can't be too big.

k=4

Valid

Not Valid

Formal definition:Prob NP iff poly time Validsuch that Prob(I) = S Valid(I,S)

•Key:•If the instance has a valid solution• A non-deterministic (fairy god mother)

could prove it to you by giving you such a solution as a witness.

• You could check that it is valid.• You could convince your boss.

k=4

Valid

•Key:•If the instance does not have a valid solution• A non-deterministic (fairy god mother)

could prove it to you by giving you ????

• You have no way to convince your boss.

k=5

•Example: 3-Col:•Instance: A graph G.•Solution: Colouring C nodes of G with 3 colours such that every edge has two colours. •G is a Yes instance if there is such a colouring. •Given an instance G and a solution C, there is a poly-time alg Valid(G,C) to test whether or not C is a valid 3- colouring of G. 3-Col NP.

Non-Deterministic Poly-Time (NP)

•Example: Airplane Wing:•Instance: Requirements I of the wing.•Solution: A description S of how to make the wing. •I is a Yes instance if there is such a wing. •Given an instance I and a proof S, there is a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.Þ Airplane Wing NP.

I = [weight, lift, cost, …]

Non-Deterministic Poly-Time (NP)

Reductions

•Reduction: Design a fast algorithm for one computational problem, using a supposedly fast algorithm for another problem as a subroutine.

Palg ≤poly Poracle

• Definition is asymmetric.• There is a witness that a graph has a 3-Col.• There is no known witness that a graph has no 3-Col.Cook Reduction: • Design any fast algorithm for Palg using a supposed fast

algorithm for Poracle as a subroutine.Karp Reduction:

The algorithm for Palg calls that for Poracle only once Yes Yes & No No

NP vs Co-NP Preserving Reductions

We will only consider reductions of this simple form. Because they preserve NP vs Co-NP

Karp Reduction: Yes Yes & No No

• Problem Pnew is NP-Complete

• Pnew not too hard.• Pnew NP

Computable

Exp

Poly

KnownGCD

NP

complete

NP-Complete Problems

Pnew

Test in poly-time if a given solution

is validSat

Computable

Exp

Poly

KnownGCD

NP

complete

NP-Complete ProblemsProblem Pnew is NP-Complete•Pnew not too hard.• Pnew NP

•Pnew sufficiently hard.• PNP, P ≤poly Pnew

• Easier: Sat ≤poly Pnew

• Cook: P ≤poly Sat Pnew

Sat

Clique: Given <G,k>, does G contains a k-clique?

A K-independent setis a set of K nodes

with no edges between them.

Independent Set: Given <G,k>, does G contains a k-Ind Set?

K-Clique vs K-Independent Set

A K-clique is a set of K nodes with all edges between them.

Clique: Given <G,k>, does G contains a k-clique?

Independent Set: Given <G,k>, does G contains a k-Ind Set?

K-Clique vs K-Independent Set

Brute Force: Try out all n choose k possible subsets.

If k = (n) then 2(n) subsets to checkIf k=3 then only O(n3)

GIVEN:Indep.

Set Oracle

<G,k>

BUILD:CliqueOracle

<G*, k>

G* has a k Indep. set or not

G has a k clique or not

Clique ≤poly Indep Set

K-Clique vs K-Independent Set

GIVEN:Indep.

Set Oracle

<G,k>

BUILD:CliqueOracle

<G*, k>

G* has a k Indep. set or not

G has a k clique or not

Clique ≤poly Indep Set

Proof of correctness:Our oracle says yes to <G,k>

iff Old oracle says yes to <G*,k>iff G* has a k indep. setiff G has a k clique

K-Clique vs K-Independent Set

G*

G

G* has edge if and only if G does not

K-Clique vs K-Independent Set

G*

G

This graph contains a

clique of size 4.

This graph contains an independent set of size 4.

if and only if

K-Clique vs K-Independent Set

•Steps for proving that Pnew is NP-Complete • 1) I am addicted to solving NP-Complete Problems• 2) I trust in my higher power to help

A witness

12 Steps

•Steps for proving that Pnew is NP-Complete • 0) Pnew NP• 1) What to reduce it to

2) What is what3) Direction of reduction & Code4) Look for similarities5) Instance Map6) Solution Map7) Valid to Valid8) Reverse Solution Map9) Valid to Valid10) Working Algorithm11) Running Time

12 Steps

Formal definition:Pnew NP iff poly time Validsuch that Pnew(I) = S Valid(I,S)

•Design poly-time algorithm Valid(I,S)•which determines whether a given solution S•is valid for a given instance I.

12 Steps Step 0: Pnew NP

Palg ≤poly Poracle

Choose a problem Pis NP-comp that is as similar to Pnew as possible and that is already known to be NP-Complete.

12 Steps Step 1: What to reduce it to

= Independent Set = CliquePnew Pis NP-comp

Inew Iis NP-comp

k=4k=3

Snew Sis NP-comp

12 Steps Step 2: What is what

Reduce Pnew to Pis NP-comp or Pis NP-comp to Pnew?

Pis NP-comp ≤poly Pnew

Palg ≤poly Poracle

12 Steps Step 3: Direction of reduction & Code

12 Steps Step 3: Direction of reduction & Code

Independent SetClique

Both instances are graphs: nodes & edgesA clique solution is a subset of the nodes with edges.An Ind Set solution is a subset of the nodes without edges.

12 Steps Step 4: Look for similarities

12 Steps Step 5: Instance Map

Ioracle = InstanceMap(Ialg)

Ialg

12 Steps Step 5: Instance Map

Ioracle

Ialgno yes

no yes

Yes mapped to YesNo to No

12 Steps Step 5: Instance Map

12 Steps Step 6: Solution Map

Ioracle

Ialg

Potential Solution S.

Potential Solution = SolutionMap(S).

12 Steps Step 6: Solution Map

Salg = SolutionMap(Soracle)

Ioracle = InstanceMap(Ialg)

Ialg

Soracle

If Soracle is valid solution for Ioracle,then Salg is valid solution for Ialg

Valid

Valid

12 Steps Step 7: Valid to Valid

Ioracle

Ialg

Potential Solution S.

Potential Solution = ReverseSolutionMap(S).

12 Steps Step 8: Reverse Solution Map

Not part of code,but of proof.

Salg

Ioracle = InstanceMap(Ialg)

Ialg

Soracle = ReverseSolutionMap(Salg)

If Salg is valid solution for Ialg,then Soracle is valid solution for Ioracle

Valid

Valid

12 Steps Step 9: Valid to Valid

Our Algalg says yes to Ialg

iff old Algoracle says yes to Ioracle

iff Ioracle has a valid solution Soracle iff Ialg has a valid solution Salg

iff Ialg is a yes instance.

Steps 6&7Steps 8&9

12 Steps Step 10: Working Algorithm

Salg = SolutionMap(Soracle)

Ioracle = InstanceMap(Ialg)

Algalg(Ialg) is poly time if the following are poly time:

Algoracle(Ioracle) Assumed

Your job

12 Steps Step 11: Running Time

Graph Colouring

Scheduling

Clique Independent Set

Palg ≤poly Poracle

or Palg Poracle

Circuit Satisfiability

Any NP-Problem

This will prove thatCir-SAT is NP-Complete.

GIVEN:Oracle for

Cir-Sat

I

BUILD:Oracle for arbitrary

NP Problem

?

I don’t even know what problem I am trying to solve!!!

ReductionAny NP-problem ≤poly Cir-SAT

We have a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.

k=4

Valid

Formal definition:Parbitrary NP iff poly time Validsuch that Parbitrary(I) = S Valid(I,S)

We need to solve some unknown NP-Problem.What do we know about it?

GIVEN:Oracle for

Cir-Sat

I

BUILD:Oracle for arbitrary

NP Problem

Please, give me a solution Ssuch that

Valid(I,S) is true.

ReductionAny NP-problem ≤poly Cir-SAT

That looks like a Turing Machine.I only know about

circuits

The Circuit Satisfiability Problem

x3x2x1

OR

ORANDAND

OR

NOTOne bit outputNo feedback

An instance is a circuit C.

F T F

A solution is an assignment X = [F,T,F…].

F F

F

F F

T

C(X) evaluates to T or F.

The Circuit Satisfiability Problem

x3x2x1

OR

ORANDAND

OR

NOT

An instance is a circuit C.A solution is an assignment X = [F,T,F…].A valid solution has C(X) = True.

F F F

F F

T

F T

FGiven a circuit, does it have a satisfying assignment?

Very Powerful

Turing (and friends) prove that any algorithm computed by aJAVA program in poly-timecan be computed by a Turing Machine in poly-time.

Cook proves that any algorithm computed by a Turing Machine in time T(n) can be computed by a family of circuits of size [T(n)]2.

But clearly, circuits compute.

The Circuit Satisfiability Problem

GIVEN:Oracle for

Cir-Sat

I

BUILD:Oracle for arbitrary

NP Problem

Please, give me a solution Ssuch that

VI(S) is true.

ReductionAny NP-problem ≤poly Cir-SAT

Thanks for the circuit.But what is S?I can only find assignments.

I build a circuit VI(S)equivalent to Valid(I,S)

GIVEN:Oracle for

Cir-Sat

I

BUILD:Oracle for arbitrary

NP Problem

Please, give me an assignment X

such that VI(X) is true.

ReductionAny NP-problem ≤poly Cir-SAT

My pleasure.Here: X

I build a circuit VI(S)equivalent to Valid(I,S)I decode X into S

and am done.

12 Step Program

•Let's be more formal.•And flow the 12 steps.

Step 0: Cir-SAT NP

•Design poly-time algorithm ValidCir-SAT(C,X)•which determines whether a given assignment X•is valid for a given circuit C.• Easy: Evaluate C(X).

Step 1: What to reduce it to

Palg ≤poly Cir-Sat

Choose a problem Pis NP-comp that is as similar to Cir-Sat as possible and that is already known to be NP-Complete.

Any NP-problem ≤poly Cir-SAT

Have none.

We have a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.

Step 2: What is what= Circuit-Sat = some NP problemPnew Parbitrary

Inew IarbitrarySnew Sarbitrary= I = S

x3x2x1

OR

ORANDAND

OR

NOT

F F F

Step 3: Direction of reduction & Code

Given oracle for Cir-Sat,we need to be able to solve any NP-Complete problem.

Cir-Sat

Parbitrary

I Sx3x2x1

OR

ORANDAND

OR

NOT

F F F

Step 4: Look for similarities

?

Step 5: Instance Map

Step 5: Instance MapI circuit We have a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.

Let Validn(I,S) be a circuit:• I is a bit string representing an instance I.• S is a bit string representing a solution S.• Outputs T if S is an encoding of a valid solution of I.

I SG,k

And over pairs of nodes

{u.v} E OR uS OR vS

Eg: Clique

|S| k

Step 5: Instance Map

Outputs T if S is an encoding of

a valid solution S of I

I S

I circuit

hard wired

Given an instance I

Circuit VI(S) = Valid(I,S)

Validn(I,S)I VI(S)

Step 6: Solution Map

Step 6: Solution MapS assignment

X is viewed as a bit string S representing a solution S.

If X is not a bit string representing a solution

then “what ever”

X=[T,F,F,T,F,T]

Outputs T if S is an encoding of

a valid solution S of I

I Shard wired

I VI(S)

Step 6: Solution MapS assignment

X is viewed as a bit string S representing a solution S.

X=[T,F,F,T,F,T]

Outputs T if S is an encoding of

a valid solution S of I

I Shard wired

I VI(S)

solution S

Step 7: Valid ValidVI(X) = T

Outputs T if S is an encoding of

a valid solution S of I

I Shard wired

I VI(S)

Valid(I,S) = Validn(I,S) = VI(X) = T

S is a valid solution of I.

Step 8: Rev. Sol. Mapsolution assignment

S bit string representing a solution S

solution S

X=[T,F,F,T,F,T]

Outputs T if S is an encoding of

a valid solution S of I

I Shard wired

I VI(S)

Step 9: Valid Valid

Outputs T if S is an encoding of

a valid solution S of I

I Shard wired

I VI(S)

S is a valid solution of I.

VI(X) = T

VI(X) = Validn(I,S) = Valid(I,S) = T

GIVEN: Alg for circuit

problem

BUILD:Opt.

problem

I

satisfiable or not

Yes/No

Reduction

VI

i.e. S, S is a valid solution for I

i.e. X, VI(X)

Any NP-problem ≤poly Cir-SAT

Graph Colouring

Scheduling

Clique Independent Set

Palg ≤poly Poracle

or Palg Poracle

Circuit Satisfiability

Any NP-Problem

3-Col?

3-Col

Graph Col: Given graph G & k can G be coloured with k colours?3-Col: Given graph G can G be coloured with 3 colours?

3-Col ≤poly Graph Col

If you can decide whether you can colour with k colours,then you can decide whether you can colour with 3 colours.

G,k C CG

Graph Colouring

Scheduling

Clique Independent Set

Palg ≤poly Poracle

or Palg Poracle

Circuit Satisfiability

Any NP-Problem

3-Col 3-SAT?

3-SAT

Cir Sat: Given circuit C does it have a satisfying assignment X?3-SAT: Given an expression:• A circuit consisting of a big AND of clauses• Each clause is the OR of at most 3 literals• Each literal is a variable or its negation.

does it have a satisfying assignment X?

xoryorz AND xorwora AND …

F T T F F F F T T F F F

3-SAT

Cir Sat: Given circuit C does it have a satisfying assignment X?3-SAT: Given an expression: does it have a satisfying assignment X?

xoryorz AND xorwora AND …

F T T F F F F T T F F F

3-Sat ≤poly Cir Sat

If you can decide whether any circuit is satifiable, then you can decide whether a circuit that happens to be an expression is satisfiable.

Graph Colouring

Scheduling

Clique Independent Set

Palg ≤poly Poracle

or Palg Poracle

Circuit Satisfiability

Any NP-Problem

3-SAT3-Col?

Graph Col ≤poly Cir SAT

»3-Col ≤poly 3-SAT

Step 5: Instance MapGraph G Expression V

Given an instance G

Circuit VG(C) = Valid(G,C)

G

Outputs T if C is an encoding of

a valid 3-colouring C of G

hard wired CEncoding C of C: u is a node r is a colour x<u,r> = T if node u is colour r.

Step 5: Instance MapGraph G Expression V

Given an instance G

Circuit VG(C) = Valid(G,C)

Ghard wired CEncoding C of C:

u is a node r is a colour x<u,r> = T if node u is colour r.And over edges <u,v> and colours r

x<u, r> = F OR x<v, r> = F

clauses with 2 literals

Step 5: Instance MapGraph G Expression V

Given an instance G

Circuit VG(C) = Valid(G,C)

Ghard wired CEncoding C of C:

u is a node r is a colour x<u,r> = T if node u is colour r. And over node u

clauses with 3 literals

x<u, r> =T OR x<u, g>=T OR x<u, b>=T