1

Desain dan Analisis Algoritma

Ilmu Komputer FMIPA UNUD

Pengampu : Luh Gede Astuti Putu Indah C.

2

DAA

Algoritma terencana, dapat dilakukan, selesai

Desain Algoritma jurus pemecahan masalah

Analisis Algoritma penentuan kelas algoritma

3

Materi Kuliah Text

Introduction to Algorithms, 2nd edition T. H. Cormen, C. E. Leiserson, R. L. Rivest, and Clifford Stein Published by: MIT Press or McGraw-Hill

Introduction to the design and analysis of algorithm Anany Levitin Published by: Addison Wesley

4

Tujuan Instruksional Umum

Setelah mengikuti kuliah ini mahasiswa akan dapat :1. Menggunakan tools dan teknik-teknik yang lazim digunakan

untuk analisis dan desain algoritma, 2. Mendesain, menganalisis, dan menentukan kebenaran suatu

algoritma terhadap kasus-kasus tertentu,3. Membandingkan beberapa algoritma dan menentukan algoritma

yang terbaik untuk memecahkan kasus-kasus tertentu.

5

Pokok Bahasan Basic algorithmic analysis

1. Asymptotic analysis of upper and average complexity bounds

2. Identifying differences among best, average, and worst case behaviors

3. Big "O," little "o," omega, and theta notation

4. Standard complexity classes

5. Time and space tradeoffs in algorithms

6. Using recurrence relations, characteristic equation, and master theorem to analyze recursive algorithms

Algorithmic strategies1. Brute-force algorithms

2. Greedy algorithms

3. Divide-and-conquer

4. Backtracking

5. Branch-and-bound

6. Heuristics

7. Pattern matching and string/text algorithms

8. Numerical approximation algorithms

9. Dynamic Programming

6

A First Step Toward Algorithm Complexity

Analysis

Design and Analysis of Algorithms

7

Contents

Algorithm Design and Analysis Process Important Problem Types The Need of Efficient Algorithm Analysis Framework

8

Algorithm Design and Analysis Process

Understand the problem

Decide on:Computational means, exact vs approximate

solving, data structure(s), algorithm design

technique

Design an algorithm

Prove correctness

Analyze the algorithm

Code the algorithm

An input instance of the problem; specify the range of instances

The capabilities of a computational device

Approximation: The problem cannot solved exactly,

exp: square root Available exact algs are unacceptably

slow The appr alg is a part of a more

sophisticated exact alg

Algorithm + Data Structures = Program

A general approach to solving problem algorithmically

9

Understand the problem

Decide on:Computational means, exact vs approximate

solving, data structure(s), algorithm design

technique

Design an algorithm

Prove correctness

Analyze the algorithm

Code the algorithm

Specifying an algorithm: Using natural language Using flowchart Using hardware design Using program source code Using pseudocode Other more convenient form?

Correctness: prove that the algorithm yields a required result for every legitimate input in a finite amount of time Usually using mathematical induction Can we use simple tracing?

Incorrectness Approx alg the error < limit

Algorithm Design and Analysis Process

10

Understand the problem

Decide on:Computational means, exact vs approximate

solving, data structure(s), algorithm design

technique

Design an algorithm

Prove correctness

Analyze the algorithm

Code the algorithm

Algorithm qualities: Correctness Efficiency:

Time efficiency Space efficiency

Simplicity Generality: the problem, input range

Programming an algorithm: Peril: incorrect / inefficient transition Program correctness proving? Practical: testing & debugging

Algorithm Design and Analysis Process

11

Important Problem Types

Sorting Searching String processing Graph problems Combinatorial problems Geometric problems Numerical problems

12

Problem Types: Sorting The problem: rearrange the item of a

given list in ascending order In case of records, we need a key There are dozens of sorting algorithms Two properties of sorting algorithms:

Stable: it preserve the relative order of any two equal elements in its input

In place: it does not require extra memory, except, possibly, for a few memory units

13

Problem Types: Searching The problem: finding a given value (search key) in a

given set Searching algorithms range:

sequential search to binary search (spectacularly efficient, but limited) and algorithm based on representing the set in a different form more conducive to search

Challenges: Very large data set Update: add, edit, delete

14

Problem Types: String Processing String = a sequence of characters from

alphabet Particular interest: text strings, binary

strings, gene sequences etc. One particular problem: string matching

Searching for a given word in a text

15

Problem Types: Graph Problems

Basic graph algorithms: graph traversal, shortest-path, topological sorting for graph with directed edges

Some problems are computationally very hard –only very small instances can be solved in a realistic amount of time–

Traveling Salesman Problem Graph-Coloring Problem

16

Problem Types: Combinatorial Problems

The problem: find a combinatorial object –such as a permutation, a combination, or a subset – that satisfies certain constraints and has some desired property

The most difficult problems The number of combinatorial objects typically grows

extremely fast with a problem’s size There are no known exact algorithms for solving such

problems in an acceptable amount of time From a more abstract perspective, TSP & GCP are

examples of combinatorial problem

17

Problem Types: Geometric Problems

Deals with geometric objects: points, lines, polygons etc.

Ancient Greek: to construct simple geometric shapes –triangles, circles etc.– with unmarked ruler and compass

Today people: application to computer graphics, robotics, tomography etc.

Classic problems: Closest-pair problem: given n points in the plane, find the

closest pair among them

18

Problem Types: Numerical Problems

Involves mathematical objects of continuous nature: solving equations and system of equations, computing definite integrals, evaluating functions etc.

The majority of such mathematical problem can only solved approximately

Computer can only represent real number approximately Accumulation of the round-off error

Computing industry focus shifting: numerical analysis (in industry & science) to business application (information storage, retrieval, transportation through network, and presentation to users)

19

The Need of Efficient Algorithm

Suppose that you have an infinitely fast computer equipped with unlimited capacity of free-memory.

Do you still have any reason to study algorithm?

20

Absolutely YES!

You still have to demonstrate that your solution method terminates and does so with the correct answer

21

Back to the real world Computers may be fast, but they are not

infinitely fast Memory may be cheap, but it is not free.

Bounded resources: Computing time Space in memory

These resources must be used wisely, and efficient algorithms will help you do so

22

Efficiency: An Illustration Pick two sorting algorithms:

Insertion sort: takes time roughly equal to c1n2 to sort n items n2

Merge sort: takes (c2 n log2 n) to sort n items n log2 n c1 < c2 far less significant than the input size n Insertion sort is usually faster than merge sort for

small input sizes. Once the input size n becomes large enough,

merge sort’s advantage of log2 n vs n will more to compensate the difference in constant factors

23

Efficiency: Concrete example (1)

Array to sort: 106 numbers Computer A: 109 inst/sec; running

insertion sort; craftiest programmer; codeIS 2n2

Computer B: 107 inst/sec; running merge sort; average programmer, HLL; the codeMS 50 n log2 n

Comp A: (2(106)2) / 109 = 2000 sec Comp B: (50.106 log2 106) / 107 ≈ 100 sec

24

Efficiency: Concrete example (2)

By using an algorithm whose running time grows more slowly, even with a poor compiler, comp B runs 20 faster than A.

Let’s try to sort 107 numbers… Comp A: timeIS ≈ 2.3 days Comp B: timeMS < 20 minutes

25

Analysis Framework

Measuring an input’s size Unit for Measuring Running Time Orders of Growth Worst-case, Best-case, and Average-case

Efficiency

26

Measuring an input’s size Almost all algorithms run longer on larger inputs It’s logical to investigate an algorithm’s efficiency as

a function of some parameter n indicating the algorithm’s input size

In most cases, selecting n is straightforward; exp: the size of the list for sorting, searching etc.

For the problem of evaluating polynomial of degree n, it will be polynomial’s degree or the number of its coefficient’s

27

The Choice of a Parameter Indicating an Input Size Does Matter

Example: computing the product of two n-by-n matrices

Two natural measures: The matrix order n The total number of elements N in the matrix

being multiplied applicable to n-by-m matrices

28

The Choice can be Influenced by Operations of the Algorithm

How should we measure an input’s size for a spell-checking algorithm?

If it examines individual characters of its input the number of characters

If it works by processing words the number of words For algorithms involving properties of numbers (e.g.

is integer n prime?) Size = number of bit b in the n’s binary representation b = log2 n + 1

29

Unit for Measuring Running Time

Can we use some standard units of time measurement –a second, a millisecond, and so on– ?

Drawbacks: dependence on the speed of a particular computer, dependence on the quality of a program, difficulty of clocking the actual running time of the program

One possible approach: to count the number of times each of the algorithm’s operations is executed difficult & unnecessary

30

Basic Operation Identify the most important operation of the

algorithm (basic operation) The operation contributing the most to the total running

time Compute the number of times the basic operation is

executed

The established framework for analysis of an algorithm’s time efficiency: counting the number of times the algorithm’s basic operation is executed on input of size n