leonghw, soc, nus (uit2201: algorithms) page 1 algorithms readings: [sg] ch. 2 & 3 chapter...
TRANSCRIPT
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 1
Algorithms
Readings: [SG] Ch. 2 & 3
Chapter Outline:1. Chapter Goals
2. What are Algorithms1. Real Life Examples (origami, recipes)
2. Definition
3. Example: A = B + C
3. Expressing Algorithms – Pseudo-Code
4. Simple Algorithms
5. Recursive Algorithms
6. Time Complexity of Algorithms
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 2
Algorithms
Computing devices are dumb How to explain to a dumb mechanical /
computing device how to solve a problem
How to solve a problem using “a small, basic set of primitive
instructions”.
Complexity of Solving Problems.
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 3
1. Goals of Algorithm Study
To develop framework for instructing computer to perform tasks
To introduce notion of algorithm as means of specifying how to solve a problem
To introduce and appreciate approaches for defining and solving very complex tasks in terms of simpler tasks;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 4
Chapter Outline:1. Chapter Goals
2. What are Algorithms1. Real Life Examples (origami, recipes)
2. Definition of Algorithm
3. Example: A = B + C
3. Expressing Algorithms – Pseudo-Code
4. Simple Algorithms
5. Recursive Algorithms
6. Time Complexity of Algorithms
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 5
2. Computer Science and Algorithms…
Computer Science can be considered…
as study of algorithms including their formal properties Their hardware and software realisations Their applications to diverse areas
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 6
Algorithms: Real Life Examples
Many Real-Life Examples Cooking: Recipe for preparing a dish Origami: The Art of Paper Folding Directions: How to go to Changi Airport
Remember: Framework: How to give instructions; Alg: The actual step-by-step instructions; Abstraction: Decomposing / Simplifying
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 7
Real Life Example: Cooking
Framework: “Cooking or Recipe language”
Algorithm: “Recipe for a Dish”
(step by step instructions on how to cook a dish)
Problem Decomposition Preparing Ingredients; Preparing Sauce;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 8
Real Life Example: Origami
Framework: “Origami or Paper-Folding language”
Algorithm: “Sequence of Paper-Folding Instructions”
(step by step instructions for each fold)
Problem Decomposition Start with a Bird Base; … Finish the Head; Finish the Legs; Finish the Tail;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 9
Real Life Examples: Issues
Problems/Difficulties: Imprecise Instructions; Job can often be done even if instructions are
not followed precisely Modifications may be done by the person
following the instructions;
But, NOT for a Computer Needs to told PRECISELY what to do;
Instructions must be PRECISE; Cannot be vague or ambiguous
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 10
Definition of Algorithm:
An algorithm for solving a problem “a finite sequence of unambiguous, executable
steps or instructions, which, if followed would ultimately terminate and give the solution of the problem”.
Note the keywords: Finite set of steps; Unambiguous; Executable; Terminates;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 11
Algorithm Examples?
Problem 1: What is the largest integerINPUT: All the integers { … -2, -1, 0, 1, 2, … }
OUTPUT: The largest integer
Algorithm: Arrange all the integers in a list in decreasing order; MAX = first number in the list; Print out MAX;
WHY is the above NOT an Algorithm? (Hint: How many integers are there?)
Problem 2: Who is the tallest women in the world? Algorithm: Tutorial...
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 12
Example: Adding two (n-digit) numbers
Input: Two positive m-digit decimal numbers (a and b)
am, am-1, …., a1
bm, bm-1, …., b1
Output: The sum c = a + b
cm+1, cm, cm-1, …., c1
(Note: In the textbook, it was am-1,…,a1,a0 …)
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 13
How to “derive” the algorithm
Adding is something we all know done it a thousand times, know it “by heart”
How do we give the algorithm? A step-by-step instruction to a dumb machine
Try an example: 3 4 9 2
8 1 5 7
“Imagine you looking at yourself solving it”
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 14
Step 1: Set the value of carry to 0
Step 2: Set the value of i to 1.
Step 3: Repeat steps 4 through 6 until the value of i is > m.
Step 4: Add ai and bi to the current value of carry, to get xi.
Step 5: If xi < 10 then
let ci=xi and reset carry to 0.
Else (* i.e. xi 10 *)
let ci=xi - 10 and reset carry to 1.
Step 6: Increase the value of i by 1.
Step 7: Set cm+1 to the value of carry.
Step 8: Print the final answer cm+1, cm, …., c1
Step 9: Stop.
Algorithm: Finding sum of A & B
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 15
Chapter Outline:1. Chapter Goals2. What are Algorithms3. Expressing Algorithms – Pseudo-Code
1. Communicating Alg to computer2. Pseudo-Code3. Primitive Operations and examples4. Variables and Arrays5. Algorithm C=A+B in pseudo-code
4. Simple Algorithms5. Recursive Algorithms6. Time Complexity of Algorithms
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 16
3. Expressing Algorithms to Computer
To communicate algorithm to computer Need way to “represent” the algorithm Cannot use English
Can use computer language machine language and programming languages (Java, Pascal, C) But, these are too tedious (&technical)
Use Pseudo-Code instead…
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 17
Pseudo-Code to express Algorithms
Pseudo-Code Mixture of computer language and English
Somewhere in between precise enough to describe what is meant without
being too tediuos
Examples: Let c be 0; Sort the list of numbers in increasing order;
Need to know both syntax – representation semantics – meaning
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 18
Primitive Operations
To describe an algorithm, we need some well-defined programming primitives Assignment primitive:
assignment statement, input/output statements
Conditional primitive: if statement case statement
Looping (iterative) primitive: for loop, while loop,
Statements are executed one-by-one
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 19
Conditional Primitives…
if statement to take different actions based on condition
Syntaxif (condition) then (Step A) else (Step B)endif
if (condition) then (Step A)endif
Semantics
condition?
Step B
true false
Step A
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 20
Examples -- (if statement)…
Let mark be the total-mark obtainedif (mark < 40) then (print “Student fail”) else (print “Student pass”)endif…
read in mark (*from the terminal*)if (mark < 40) then (Grade “F”) else if (mark < 50) then (Grade “D”) else if (mark < 60) then (Grade “C”) else if (mark < 70) then (Grade “B”) else if (mark < 80) then (Grade “A”);endifprint “Student grade is”, Grade…
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 21
the while-loop loop a “variable”
number of times
Syntaxwhile (condition) do
(some sequence
of statements)
endwhile
Semantics…
Looping Primitive – while-loop
condition?
Some sequenceof statements;
true
false
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 22
First, the for-loop loop a “fixed” or (pre-
determined) number of times
Syntaxfor j a to b do
(some sequence
of statements)
endfor
Semantics…
Looping Primitive – for-loop
j a;
(j <= b)?
Some sequenceof statements;
j j+1;
false
true
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 23
“Exercising the alg”: for and while
for j 1 to 4 do print 2*j;endforprint “--- Done ---”
Output: 2 4 6 8--- Done ---
j 1;while (j <= 4) do print 2*j; j j + 1;endwhileprint “--- Done ---”
Output: 2 4 6 8--- Done ---
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 24
Variables and Arrays…
In the computer, each variable is assigned a storage “box” can store one number at any time eg: sum, j, carry
Arrays: Often deal with many numbers Such as A1, A2, A3, … , A100
Store as an “array” A[1], A[2], … , A[100] we treat each of them as a variable, each is assigned a storage “box”
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 25
Algorithm: A = B + C (in pseudo-code)
We can re-write the C=A+B algorithm as follows:
Alg. to Compute C = A + B: (*sum two big numbers*)carry 0;for i 1 to m do
x[i] a[i] + b[i] + carry ;if (x[i] < 10) then ( c[i] x[i]; carry 0; )
else ( c[i] x[i] – 10; carry 1; )endfor;c[m+1] carry;
Print c[m+1], c[m], …., c[1]
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 26
Chapter Outline:1. Chapter Goals
2. What are Algorithms
3. Expressing Algorithms – Pseudo-Code
4. Simple Algorithms1. Simple iterative algorithms
Computing Sum, Find Max/Min
2. Modular Program Design
3. Divisibility and Prime Numbers
5. Recursive Algorithms
6. Time Complexity of Algorithms
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 27
Simple iterative algorithm: Sum
Given: Given: List of numbers: A1, A2, A3, …., An
Output: To compute the sum of the numbers
Note: Store numbers in array A[1], A[2], … , A[n]
Sum(A, n);begin Sum_sf 0; k 1; while (k <= n) do Sum_sf Sum_sf + A[k]; k k + 1; endwhile Sum Sum_sf; Print “Sum is”, Sumend;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 28
Exercising Algorithm Sum:
A[1] A[2] A[3] A[4] A[5] A[6] n=6 2 5 10 3 12 24
k Sum-sf Sum ? 0 ? 1 2 ? 2 7 ? 3 17 ? 4 20 ? 5 32 ? 6 56 ? 6 56 56
Sum is 56
Input:
Processing:
Output:
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 29
Algorithm for Sum (with for-loop)
We can also use a while-loop instead of a for loop.
HW: (a) Note the differences… (b) Modify it to compute the average?
Sum(A, n);(* Find the sum of A1, A2,…, An. *)begin Sum_sf 0; for k 1 to n do Sum_sf Sum_sf + A[k]; endfor Sum Sum_sf; Print “Sum is”, Sumend;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 30
Remarks about the iterative algorithm…
Note the three stages:1. Initialization
Set some values at the beginning
2. Iteration This is the KEY STEP Where most of work is done
3. Post-Processing or Cleanup
Can use this setup for other problems Calculating average, sum-of-squares Finding max, min; Searching for a number,
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 31
Modular Program Design
Software are complex HUGE (millions of lines of code) eg: Linux, Outlook COMPLEX; eg: Flight simulator
Idea: Divide-and-Conquer Method Complex tasks can be divided and each part solved
separately and combined later.
Modular Program Design Divide big programs into smaller modules The smaller parts are
called modules, subroutines, or procedures Design, implement, and test separately
Modularity, Abstraction, Division of Labour Simplifies process of writing alg/programs
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 32
Simple Algorithms: Prime Numbers
Algorithm to determine if n is primeGiven: A positive integer n
Question: Is n a prime number?
What do we know?“A number n is prime iffn has no positive factors except 1 and n”
Idea: Express it “algorithmically”“n is not divisible by any positive number k such that 1 < k < n.” or k=2,3,4,…,(n-1)
What we already have: A module Divisible(m,n) to check divisibility We can use it as a primitive
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 33
Pseudo-Code for Prime:
Note: Prime uses the module Divisible(k,n) Exercise it with:
Prime (5); Prime (4); Prime (55); Prime (41);
Prime(n)(* To determine if n is prime *)begin for k 2 to (n-1) do if Divisible(k,n) then (Print “False” & exit) else (* Do nothing *) endfor Print “True” & Stopend;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 34
A Simple Module: Divisibility
A module (procedure) for divisibilityGiven: Two positive integers m and n
Question: Is n divisible by m?
or Algorithm to compute Divisible(m,n)
Algorithm Idea:“A positive integer n is divisible by m iff for some positive integer k n, m*k = n.”
mby divisiblenot isn if False,
mby divisible isn if True,),( {nmDivisible
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 35
Module for Divisibility:
Algorithm (in pseudo-code)
Exercise it with: Divisible (3, 9); Divisible (3, 5); Divisible (3,101);
Divisible(m,n)(* To compute if n is divisible by m *)begin D false; (* assume false, first *) for k 1 to n do if (m*k = n) then (Dtrue; exit-loop) else (* Do nothing *) endfor Divisible D; (* true or false *)end;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 36
Chapter Outline:1. Chapter Goals
2. What are Algorithms
3. Expressing Algorithms – Pseudo-Code
4. Simple Algorithms
5. Recursive Algorithms
6. Time Complexity of Algorithms1. Sequential search algorithm
2. Binary search algorithm
3. Analysis of Time Complexity
7. Summary…
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 37
Search: sequential search algorithm
Given: Given: List of numbers: A1, A2, A3, …., An and a query number x;
Question: Search for x in the list;
Sequential-Search(A, n, x);(* Search for x in A1, A2,…, An. *)begin for k 1 to n do if (x = A[k]) then (Print “Yes”; Stop) else (* do nothing *) endfor Print “No”; Stopend;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 38
Remarks on Sequential Search Alg
Analogy: House-to-house search…
How fast is it to search for x? How many comparisons do we need? Example: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55
When x=14, need 6 comparisons When x=13, need only 1 comparison BEST CASE When x=55, need 10 comparisons WORST CASE When x=5, need 10 comparisons WORST CASE
In general, given n numbers, A1,…,An
Best Case: 1 comparison Worst Case: n comparisons Average Case: (n+1)/2 comparisons
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 39
Binary Search
If the List is sorted, that is
A1 A2 A3 …. An
Then, we can do better, actually a lot better….
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 40
Binary Search
1, 4, 9, 11, 14, 43, 78
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 41
Binary Search (How fast is it?)
Imagine n=100 After 1 step, size is 50 After 2 steps, size is 25 After 3 steps, size is 12 After 4 steps, size is 6 After 5 steps, size is 3 After 6 steps, size is 1 DONE!!
What if n=1000? 1,000,000?
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 42
Binary Search Algorithm
Input: Sorted List A1 A2 A3 …. An
A number x.
Question: Is x in the list?
Binary-Search(A,n,x);
1. First 1
Last n
2. While ( First Last ) do
mid (first+last) / 2 If ( x = A[mid] )
then (output Yes and Stop)
else If ( x < A[mid] )
then Last mid-1
else If ( x > A[mid] ) then First mid+1
EndWhile
3. Output False and Stop
4. End
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 43
Time Complexity Analysis
Sequential Search (Alg): Worst Case: n comparisons Best Case: 1 comparison Avg Case: n/2 comparisons
Binary Search (Alg): Worst Case: log2 n comparisons Best Case: 1 comparison Avg Case: about log2 n comparisons
How to get the Average Case? using mathematical analysis…
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 44
Complexity of Algorithm…
Logarithmic Time Algorithm Binary Search
A Linear Time Algorithm Algorithm Sum(A,n) -- O(n) time Algorithm Sequential-Search(A,n,x) – O(n) time
A Quadratic Time Algorithm Simple Median-Find (T2-Q3)
An Exponential Time Algorithm All-Subsets(A,n)
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 45
Chapter Outline:1. Chapter Goals
2. What are Algorithms
3. Expressing Algorithms – Pseudo-Code
4. Simple Algorithms
5. Recursive Algorithms1. Recursion – the idea
2. Fibonacci sequence
3. Tower of Hanoi
6. Time Complexity of Algorithms
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 46
5. Recursion
A problem solving method of “decomposing bigger problems into smaller sub-problems that are identical to itself.”
General Idea: Solve simplest (smallest) cases DIRECTLY
usually these are very easy to solve Solve bigger problems using smaller sub-problems
that are identical to itself (but smaller and simpler)
Abstraction: To solve a given problem, we first assume that we
ALREADY know how to solve it for smaller instances!!
Dictionary definition:recursion
see recursion
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 47
Example: Fibonacci Numbers… Definition of Fibonacci numbers
1. F1 = 1,
2. F2 = 1,
3. for n>2, Fn = Fn-1 + Fn-2
Problem: Compute Fn for any n. The above is a recursive definition.
Fn is computed in-terms of itself actually, smaller copies of itself – Fn-1 and Fn-2
Actually, Not difficult:F3 = 1 + 1 = 2 F6 = 5 + 3 = 8 F9 = 21 + 13 = 34
F4 = 2 + 1 = 3 F7 = 8 + 5 = 13 F10 = 34 + 21 = 55
F5 = 3 + 2 = 5 F8 = 13 + 8 = 21F11 = 55 + 34 = 89
1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 48
Fibonacci Numbers: Recursive alg
The above is a recursive algorithm
It is simple to understand and elegant!
But, very SLOW
Fibonacci(n) (* Recursive, SLOW *)begin if (n=1) or (n=2) then Fibonacci(n) 1 (*simple case*) else Fibonacci(n) Fibonacci(n-1) + Fibonacci(n-2) endifend;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 49
Recursive Fibonacci Alg -- Remarks
How slow is it? Eg: To compute F(6)…
F(5)
F(4) F(3)
F(3) F(2)
F(2) F(1)
F(2) F(1)
F(4)
F(3) F(2)
F(2) F(1)
F(6)
HW: Can we compute it faster?
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 50
Given: Three Pegs A, B and CPeg A initially has n disks, different size, stacked up,
larger disks are below smaller disks
Problem: to move the n disks to Peg C, subject to1. Can move only one disk at a time
2. Smaller disk should be above larger disk
3. Can use other peg as intermediate
Example: Tower of Hanoi
A B C A B C
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 51
Tower of Hanoi
How to Solve: Strategy… Generalize first: Consider n disks for all n 1 Our example is only the case when n=4
Look at small instances… How about n=1
Of course, just “Move disk 1 from A to C”
How about n=2?1. “Move disk 1 from A to B”
2. “Move disk 2 from A to C”
3. “Move disk 1 from B to C”
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 52
Tower of Hanoi (Solution!)
General Method: First, move first (n-1) disks from A to B Now, can move largest disk from A to C Then, move first (n-1) disks from B to C
Try this method for n=3
1. “Move disk 1 from A to C”2. “Move disk 2 from A to B”3. “Move disk 1 from C to B”
4. “Move disk 3 from A to C”
5. “Move disk 1 from B to A”6. “Move disk 1 from B to C”7. “Move disk 1 from A to C”
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 53
Algorithm for Towel of Hanoi (recursive)
Recursive Algorithm when (n=1), we have simple case Else (decompose problem and make recursive-calls)
Hanoi(n, A, B, C);(* Move n disks from A to C via B *)begin if (n=1) then “Move top disk from A to C” else (* when n>1 *) Hanoi (n-1, A, C, B); “Move top disk from A to C” Hanoi (n-1, B, C, A); endifend;
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 54
Characteristics of Algorithm
Correctness
Complexity --- time, space (memory), etc
Ease of understanding
Ease of coding
Ease of maintainence
LeongHW, SoC, NUS(UIT2201: Algorithms) Page 55
If you are new to algorithms read the textbook try out the algorithms do the exercises
… The End …