Algorithms and Data ReviewFall 2010

Karl Lieberherr

1CS 4800 Fall 201012/7/2010

Problem Solving

• Polya: How to Solve It

2CS 4800 Fall 201012/7/2010

Algorithmic Patterns

• Greedy• Divide and Conquer• Dynamic Programming• Network Flow• Intractability: Reductions and characterization

of NP

3CS 4800 Fall 201012/7/2010

Algorithmic thinking

• Making claims about algorithms• Choosing from a predefined set of claims

– claims that are refuted– claims that are defended

• Used: Scientific Community Game for Algorithmic domains– make learning interactive and second order– students who are better at the game are better at

algorithms12/7/2010 CS 4800 Fall 2010 4

Summary of SCG Definitions

Domain Problem Solution valid(Problem, Solution) quality(Problem, Solution) →[0,1]

5CS 4800 Fall 2010

Claim(Domain) Problems: PowerSet(Domain.Problem) q: Quality = [0,1] r: Resource = N+

Rules of the Scientific Community: propose and oppose,be an active scholar, rules for reputation accumulation.

Tournaments

12/7/2010

Formal: HSR

• Domain: – Problem: (n,k), k <= n.– Solution: Decision tree to determine highest safe

rung.– quality(problem, solution): depth of decision tree /

number of rungs– valid(problem, solution): at most k left branches, ...

6CS 4800 Fall 201012/7/2010

Formal: HSR

• Claim(Domain): – Alice claims ({(25,2)},9/25,5 seconds)

• {(25,2)}: set of problems (singleton)• 9/25: quality• 5 seconds: resource

• Refutation Protocol:– Bob refutes: only one problem: (25,2)– Alice: solves problem by providing decision tree t.– predicate: t is a valid decision tree for (25,2) of depth 9

CS 4800 Fall 2010 712/7/2010

Protocol variationsExample: Independent Set

• Domain– Problem: Directed graph G=(V,E)– Solution: V’ subset of V

• Claim– Problems: all directed graphs with < 10^6 nodes– quality: |V’|/|V|

• Protocol– Alice to Admin: G, AlgA(G)– Admin to Bob: G– Bob to Admin: AlgB(G)– Admin to both: AlgB(G), r[G,AlgB(G),AlgA(G)]

12/7/2010 CS 4800 Fall 2010 8

Alice claims that her algorithm findsa solution at least as good as Bob’s.

Protocol variationsExample: Knapsack

• Domain– Problem: Knapsack problem K(n,Items(Weights,Values),W)– Solution: subset of Items

• Claim– Problems: all knapsack problems with W < 10^3 – quality: |V’|/|V|

• Protocol– Alice to Admin: K, AlgA(K)– Admin to Bob: K– Bob to Admin: AlgB(K) including AlgB(K).resource– Admin to both: AlgB(K) including AlgB(K).resource,

r[K,AlgB(K),AlgA(K)]

12/7/2010 CS 4800 Fall 2010 9

Alice claims that her algorithm findsa solution at least as good as Bob’sand using fewer resources.

Languages for expressing algorithms

• Decision trees• Finite state machines• Pseudo code• Programming languages

10CS 4800 Fall 201012/7/2010

Prove properties about algorithm

• Find an invariant of the algorithm that helps to prove what you want.

• Example: Ford-Fulkerson• Dynamic Programming: formulate recurrence

for optimal solution in terms of optimal solutions for sub problems.

• Formulate algorithm so that it is easier to prove its correctness.

11CS 4800 Fall 201012/7/2010

Intractability

• Problems that don’t have an algorithmic solution. E.g. is grammar ambiguous?

• Problems that don’t have an efficient algorithmic solution, e.g., NP-complete problems.

• Reduction: Y <= X : Y is reducible to X, Y reduces to X: an algorithm for X implies an algorithm for Y

• Polynomial Reduction: Y <=p X: a polynomial algorithm for X implies a polynomial algorithm for Y.

12CS 4800 Fall 201012/7/2010

NP-complete Problem X

1. X in NP2. for all Y in NP, Y <=p X.

Every problem in NP can be reduced to X.

Circuit Satisfiability is NP-complete.

13CS 4800 Fall 201012/7/2010

More NP-complete problems

• If Y is NP-complete and X is in NP with the property that Y <=p X, then X is NP-complete.

14CS 4800 Fall 201012/7/2010

Strategy for proving problem X to be NP-complete

• Prove that X in NP.• Choose a problem Y that is NP-complete.• Prove that Y <=p X.• Cook and Karp (translation) reductions (page

473)• Example of Cook reduction: search (FSAT)

reduces to decision (SAT)

15CS 4800 Fall 201012/7/2010

Interview question

• Given a set S and a predicate pred defined on subsets of S, find a subset ss of S satisfying pred.– Example pred: false on all subsets not having 3

elements. For 3 element subsets pred({a,b,c}) iff a+b+c=0.

16CS 4800 Fall 201012/7/2010

Order Statistics

• Given a sequence of n numbers, find the kth largest element.

• O(n*log(n))• O(n*log(k))• O(n): median of the median

17CS 4800 Fall 201012/7/2010

Expected number of satisfied constraints under random assignment• Maximum Satisfiability• Maximum Boolean CSP

18CS 4800 Fall 201012/7/2010

Problem Snapshot• Boolean CSP: constraint satisfaction problem

– Each constraint uses a Boolean relation.– e.g. a Boolean relation 1in3(x y z) is satisfied iff

exactly one of its parameters is true.

• Boolean MAX-CSP a multi-set of constraints. Maximize satisfied fraction.

19CS 4800 Fall 201012/7/2010

1in3

0

0.1

0.2

0.3

0.4

0.5

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Coin bias (Probability of setting a variable to true)

Fra

ctio

n o

f co

nst

rain

ts t

hat

ar

e g

uar

ante

ed t

o b

e sa

tisf

ied

3p(1-p)2 for MAX-CSP({22})

20CS 4800 Fall 201012/7/2010

all the look-ahead polynomials for T Ball

21CS 4800 Fall 201012/7/2010